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PREFACE 


This  book  provides  a  basic,  conceptual-level  description  of  engineering  management  disciplines  that 
relate  to  the  development  and  life  cycle  management  of  a  system.  For  the  non-engineer  it  provides  an 
overview  of  how  a  system  is  developed.  For  the  engineer  and  project  manager  it  provides  a  basic  frame¬ 
work  for  planning  and  assessing  system  development. 

Information  in  the  book  is  from  various  sources,  but  a  good  portion  is  taken  from  lecture  material 
developed  for  the  two  Systems  Planning,  Research,  Development,  and  Engineering  courses  offered  by 
the  Defense  Acquisition  University. 

The  book  is  divided  into  four  parts:  Introduction;  Systems  Engineering  Process;  Systems  Analysis  and 
Control;  and  Planning,  Organizing,  and  Acquisition.  The  first  part  introduces  the  basic  concepts  that 
govern  the  systems  engineering  process  and  how  those  concepts  fit  the  DoD  acquisition  process.  Chap¬ 
ter  1  establishes  the  basic  concept  and  introduces  terms  that  will  be  used  throughout  the  book.  The 
second  chapter  goes  through  a  typical  acquisition  life  cycle  showing  how  systems  engineering  supports 
acquisition  decision  making. 

The  second  part  introduces  the  systems  engineering  problem-solving  process,  and  discusses  in  basic 
terms  some  traditional  techniques  used  in  the  process.  An  overview  is  given,  and  then  the  process  of 
requirements  analysis,  functional  analysis  and  allocation,  design  synthesis,  and  verification  is  explained 
in  some  detail.  This  part  ends  with  a  discussion  of  the  documentation  developed  as  the  finished  output 
of  the  systems  engineering  process. 

Part  three  discusses  analysis  and  control  tools  that  provide  balance  to  the  process.  Key  activities  (such  as 
risk  management,  configuration  management,  and  trade  studies)  that  support  and  run  parallel  to  the 
system  engineering  process  are  identified  and  explained. 

Part  four  discusses  issues  integral  to  the  conduct  of  a  systems  engineering  effort,  from  planning  to 
consideration  of  broader  management  issues. 

In  some  chapters  supplementary  sections  provide  related  material  that  shows  common  techniques  or 
policy-driven  processes.  These  expand  the  basic  conceptual  discussion,  but  give  the  student  a  clearer 
picture  of  what  systems  engineering  means  in  a  real  acquisition  environment. 


DSMC  wishes  to  thank  Mr.  John  Leonard,  the  principal  author  of  this  document,  and  the 
staff  of  DSMC  for  their  combined  efforts  in  developing  and  improving  this  text. 
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CHAPTER  1 


INTRODUCTION  TO 
SYSTEMS  ENGINEERING 
MANAGEMENT 


1.1  PURPOSE 

The  overall  organization  of  this  text  is  described 
in  the  Preface.  This  chapter  establishes  some  of 
the  basic  premises  that  are  expanded  throughout 
the  book.  Basic  terms  explained  in  this  chapter 
are  the  foundation  for  following  definitions.  Key 
systems  engineering  ideas  and  viewpoints  are 
presented,  starting  with  a  definition  of  a  system. 

1.2  DEFINITIONS 
A  System  Is  ... 

Simply  stated,  a  system  is  an  integrated  compos¬ 
ite  of  people,  products,  and  processes  that  provide 
a  capability  to  satisfy  a  stated  need  or  objective. 

Systems  Engineering  Is... 

Systems  engineering  consists  of  two  significant 
disciplines:  the  technical  knowledge  domain  in 
which  the  systems  engineer  operates,  and  systems 
engineering  management.  This  book  focuses  on 
the  process  of  systems  engineering  management. 

Three  commonly  used  definitions  of  systems 
engineering  are  provided  by  the  best  known  tech¬ 
nical  standards  that  apply  to  this  subject.  They  all 
have  a  common  theme: 

•  A  logical  sequence  of  activities  and  decisions 
that  transforms  an  operational  need  into  a  de¬ 
scription  of  system  performance  parameters 
and  a  preferred  system  configuration.  (MIL- 
STD-499A,  Engineering  Management,  1  May 
1974.  Now  cancelled.) 


•  An  interdisciplinary  approach  that  encom¬ 
passes  the  entire  technical  effort,  and  evolves 
into  and  verifies  an  integrated  and  life  cycle 
balanced  set  of  system  people,  products,  and 
process  solutions  that  satisfy  customer  needs. 
(EIA  Standard  IS-632,  Systems  Engineering, 
December  1994.) 

•  An  interdisciplinary,  collaborative  approach 
that  derives,  evolves,  and  verifies  a  life-cycle 
balanced  system  solution  which  satisfies  cus¬ 
tomer  expectations  and  meets  public  accept¬ 
ability.  (IEEE  P1220,  Standard  for  Applica¬ 
tion  and  Management  of  the  Systems  Engi¬ 
neering  Process,  [Final  Draft],  26  September 
1994.) 

In  summary,  systems  engineering  is  an  interdisci¬ 
plinary  engineering  management  process  that 
evolves  and  verifies  an  integrated,  life-cycle  bal¬ 
anced  set  of  system  solutions  that  satisfy  cus¬ 
tomer  needs. 

Systems  Engineering  Management  Is... 

As  illustrated  by  Figure  1-1,  systems  engineering 
management  is  accomplished  by  integrating  three 
major  activities: 

•  Development  phasing  that  controls  the  design 
process  and  provides  baselines  that  coordinate 
design  efforts, 

•  A  systems  engineering  process  that  provides  a 
structure  for  solving  design  problems  and  track¬ 
ing  requirements  flow  through  the  design  effort, 
and 
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Figure  1-1.  Three  Activities  of  Systems  Engineering  Management 


•  Life  cycle  integration  that  involves  custom¬ 
ers  in  the  design  process  and  ensure  that  the 
system  developed  is  viable  throughout  its  life. 

Each  one  of  these  activities  is  necessary  to 
achieve  proper  management  of  a  development 
effort.  Phasing  has  two  major  purposes:  it  con¬ 
trols  the  design  effort  and  is  the  major  connec¬ 
tion  between  the  technical  management  effort 
and  the  overall  acquisition  effort.  It  controls  the 
design  effort  by  developing  design  baselines  that 
govern  each  level  of  development.  It  interfaces 
with  acquisition  management  by  providing  key 
events  in  the  develop-ment  process,  where  de¬ 
sign  viability  can  be  assessed.  The  viability  of 
the  baselines  developed  is  a  major  input  for  ac¬ 
quisition  management  milestone  decisions.  As  a 
result,  the  timing  and  coordination  between  tech¬ 
nical  development  phasing  and  the  acquisition 
schedule  is  critical  to  maintain  a  healthy  acqui¬ 
sition  program. 


architectures,  and  configuration  baselines.  The 
discipline  of  this  process  provides  the  control  and 
traceability  to  develop  solutions  that  meet  cus¬ 
tomer  needs.  The  systems  engineering  process 
may  be  repeated  one  or  more  times  during  any 
phase  of  the  development  process. 

Life  cycle  integration  is  necessary  to  ensure  that 
the  design  solution  is  viable  throughout  the  life  of 
the  system.  It  includes  the  planning  associated  with 
product  and  process  development,  as  well  as  the 
integration  of  multiple  functional  concerns  into  the 
design  and  engineering  process.  In  this  manner, 
product  cycle-times  can  be  reduced,  and  the  need 
for  redesign  and  rework  substantially  reduced. 

1.3  DEVELOPMENT  PHASING 

Development  usually  progresses  through  distinct 
levels  or  stages: 


The  systems  engineering  process  is  the  heart  of  •  Concept  level,  which  produces  a  system 
systems  engineering  management.  Its  purpose  concept  description  (usually  described  in  a 

is  to  provide  a  structured  but  flexible  process  concept  study); 

that  transforms  requirements  into  specifications, 
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•  System  level,  which  produces  a  system 
description  in  performance  requirement  terms; 
and 

•  Subsystem/Component  level,  which  produces 
first  a  set  of  subsystem  and  component  prod¬ 
uct  performance  descriptions,  then  a  set  of 
corresponding  detailed  descriptions  of  the 
products’  characteristics,  essential  for  their 
production. 

The  systems  engineering  process  is  applied  to  each 
level  of  system  development,  one  level  at  a  time, 
to  produce  these  descriptions  commonly  called 
configuration  baselines.  This  results  in  a  series  of 
configuration  baselines,  one  at  each  development 
level.  These  baselines  become  more  detailed  with 
each  level. 

In  DoD  the  configuration  baselines  are  called 
the  functional  baseline  for  the  system-level 
description,  the  allocated  baseline  for  the  sub¬ 
system/component  performance  descriptions,  and 
the  product  baseline  for  the  subsystem/  compo¬ 
nent  detail  descrip-tions.  Figure  1-2  shows  the 
basic  relationships  between  the  baselines.  The 


triangles  represent  baseline  control  decision 
points,  and  are  usually  referred  to  as  technical 
reviews  or  audits. 

Levels  of  Development  Considerations 

Significant  development  at  any  given  level  in  the 
system  hierarchy  should  not  occur  until  the  con¬ 
figuration  baselines  at  the  higher  levels  are  con¬ 
sidered  complete,  stable,  and  controlled.  Reviews 
and  audits  are  used  to  ensure  that  the  baselines  are 
ready  for  the  next  level  of  development.  As  will 
be  shown  in  the  next  chapter,  this  review  and  audit 
process  also  provides  the  necessary  assessment 
of  system  maturity,  which  supports  the  DoD 
Milestone  decision  process. 


1.4  THE  SYSTEMS  ENGINEERING 
PROCESS 

The  systems  engineering  process  is  a  top-down 
comprehensive,  iterative  and  recursive  problem 
solving  process,  applied  sequentially  through  all 
stages  of  development,  that  is  used  to: 


Concept  Studies 

1 

DESIGN  DEFINITION 

System  Definition 

(Functional  Baseline) 

i  ■  • 

DESIGN  DEFINITION 

Preliminary  Design 

* 

(Allocated  Baseline) 

▲ 

\ 

DESIGN  DEFINITION 

Detail  Design 

t  . . 

(Product  Baseline) 

Figure  1-2.  Development  Phasing 
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•  Transform  needs  and  requirements  into  a  set 
of  system  product  and  process  descriptions 
(adding  value  and  more  detail  with  each  level 
of  development), 

•  Generate  information  for  decision  makers,  and 

•  Provide  input  for  the  next  level  of  development. 

As  illustrated  by  Figure  1-3,  the  fundamental  sys¬ 
tems  engineering  activities  are  Requirements 
Analysis,  Functional  Analysis/Allocation,  and 
Design  Synthesis,  all  balanced  by  techniques  and 
tools  collectively  called  System  Analysis  and  Con¬ 
trol.  Systems  engineering  controls  are  used  to  track 
decisions  and  requirements,  maintain  technical 
baselines,  manage  interfaces,  manage  risks,  track 
cost  and  schedule,  track  technical  performance, 
verify  requirements  are  met,  and  review/audit  the 
progress. 

During  the  systems  engineering  process  archi¬ 
tectures  are  generated  to  better  describe  and 


understand  the  system.  The  word  “architecture” 
is  used  in  various  contexts  in  the  general  field  of 
engineering.  It  is  used  as  a  general  description 
of  how  the  subsystems  join  together  to  form  the 
system.  It  can  also  be  a  detailed  description  of 
an  aspect  of  a  system:  for  example  the  Opera¬ 
tional,  System,  and  Technical  Architectures  used 
in  C4ISR  and  software  intensive  developments. 
However,  Systems  Engineering  Management  as 
developed  in  DoD  recognizes  three  universally 
usable  architectures  that  describe  important  as¬ 
pects  of  the  system:  functional,  physical,  and 
system  architectures.  This  book  will  focus  on 
these  architectures  as  necessary  components  of 
the  systems  engineering  process. 

The  Functional  Architecture  identifies  and  struc¬ 
tures  the  allocated  functional  and  performance 
requirements.  The  Physical  Architecture  depicts 
the  system  product  by  showing  how  it  is  broken 
down  into  subsystems  and  components.  The 
System  Architecture  identifies  all  the  products 
(including  enabling  products)  that  are  necessary 
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Figure  1-3.  The  Systems  Engineering  Process 
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to  support  the  system  and,  by  implication,  the 
processes  necessary  for  development,  produc¬ 
tion/construction,  deployment,  operations,  sup¬ 
port,  disposal,  training,  and  verification. 

Life  Cycle  Integration 

Life  cycle  integration  is  achieved  through  inte¬ 
grated  development — that  is,  concurrent  consid¬ 
eration  of  all  life  cycle  needs  during  the  develop¬ 
ment  process.  DoD  policy  requires  integrated 
development,  called  Integrated  Product  and  Prod¬ 
uct  Development  (IPPD)  in  DoD,  to  be  practiced 
at  all  levels  in  the  acquisition  chain  of  command 
as  will  be  explained  in  the  chapter  on  IPPD.  Con¬ 
current  consideration  of  all  life  cycle  needs  can  be 
greatly  enhanced  through  the  use  of  interdiscipli¬ 
nary  teams.  These  teams  are  often  referred  to  as 
Integrated  Product  Teams  (IPTs). 

The  objective  of  an  Integrated  Product  Team  is  to: 

•  Produce  a  design  solution  that  satisfies  initially 
defined  requirements,  and 

•  Communicate  that  design  solution  clearly, 
effectively,  and  in  a  timely  manner. 

Multi-functional,  integrated  teams: 

•  Place  balanced  emphasis  on  product  and 
process  development,  and 

•  Require  early  involvement  of  all  disciplines 
appropriate  to  the  team  task. 

Design-level  Integrated  Product  Team  members  are 
chosen  to  meet  the  team  objectives  and  generally 
have  distinctive  competence  in: 

•  Technical  management  (systems  engineering); 

•  Life  cycle  functional  areas  (eight  primary 
functions); 

•  Technical  specialty  areas,  such  as  safety,  risk 
management,  quality,  etc.;  or 


•  When  appropriate,  business  areas  such  as 
finance,  cost/budget  analysis,  and  contracting. 

Life  Cycle  Functions 

Life  cycle  functions  are  the  characteristic  actions 
associated  with  the  system  life  cycle.  As  illustrated 
by  Figure  1-4,  they  are  development,  production 
and  construction,  deployment  (fielding),  operation, 
support,  disposal,  training,  and  verification.  These 
activities  cover  the  “cradle  to  grave”  life  cycle  pro¬ 
cess  and  are  associated  with  major  functional 
groups  that  provide  essential  support  to  the  life 
cycle  process.  These  key  life  cycle  functions  are 
commonly  referred  to  as  the  eight  primary  func¬ 
tions  of  systems  engineering. 

The  customers  of  the  systems  engineer  perform 
the  life-cycle  functions.  The  system  user’s  needs 
are  emphasized  because  their  needs  generate  the 
requirement  for  the  system,  but  it  must  be  remem¬ 
bered  that  all  of  the  life-cycle  functional  areas 
generate  requirements  for  the  systems  engineer¬ 
ing  process  once  the  user  has  established  the 
basic  need.  Those  that  perform  the  primary 
functions  also  provide  life-cycle  representation 
in  design-level  integrated  teams. 

Primary  Function  Definitions 

Development  includes  the  activities  required  to 
evolve  the  system  from  customer  needs  to  product 
or  process  solutions. 

Production  and  Construction  includes  the  fabri¬ 
cation  of  engineering  test  models  and  “brass- 
boards,”  low-rate  initial  production,  full-rate 
production  of  systems  and  end  items,  or  the  con¬ 
struction  of  large  or  unique  systems  or  subsystems. 

Deployment  (Fielding)  includes  the  activities  nec¬ 
essary  to  initially  deliver,  transport,  receive,  pro¬ 
cess,  assemble,  install,  checkout,  train,  operate, 
house,  store,  or  field  the  system  to  achieve  full 
operational  capability. 

Operation  is  the  user  function  and  includes 
activities  necessary  to  satisfy  defined  operational 
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Figure  1-4.  Primary  Life  Cycle  Functions 


objectives  and  tasks  in  peacetime  and  wartime 
environments. 

Support  includes  the  activities  necessary  to  pro¬ 
vide  operations  support,  maintenance,  logistics, 
and  material  management. 

Disposal  includes  the  activities  necessary  to  ensure 
that  the  disposal  of  decommissioned,  destroyed, 
or  irreparable  system  components  meets  all 
applicable  regulations  and  directives. 

Training  includes  the  activities  necessary  to 
achieve  and  maintain  the  knowledge  and  skill 
levels  necessary  to  efficiently  and  effectively 
perform  operations  and  support  functions. 

Verification  includes  the  activities  necessary  to 
evaluate  progress  and  effectiveness  of  evolving 
system  products  and  processes,  and  to  measure 
specification  compliance. 


Systems  Engineering  Considerations 

Systems  engineering  is  a  standardized,  disciplined 
management  process  for  development  of  system 
solutions  that  provides  a  constant  approach  to 
system  development  in  an  environment  of  change 
and  uncertainty.  It  also  provides  for  simultaneous 
product  and  process  development,  as  well  as  a 
common  basis  for  communication. 

Systems  engineering  ensures  that  the  correct 
technical  tasks  get  done  during  development 
through  planning,  tracking,  and  coordinating. 
Responsibilities  of  systems  engineers  include: 

•  Development  of  a  total  system  design  solution 
that  balances  cost,  schedule,  performance,  and 
risk; 

•  Development  and  tracking  of  technical 
information  needed  for  decision  making; 

•  Verification  that  technical  solutions  satisfy 
customer  requirements; 


8 


Chapter  1 


Introduction  to  Systems  Engineering 


•  Development  of  a  system  that  can  be  pro¬ 
duced  economically  and  supported  through¬ 
out  the  life  cycle; 

•  Development  and  monitoring  of  internal  and 
external  interface  compatibility  of  the  system 
and  subsystems  using  an  open  systems 
approach; 

•  Establishment  of  baselines  and  configuration 
control;  and 

•  Proper  focus  and  structure  for  system  and  major 
sub-system  level  design  IPTs. 

1.5  GUIDANCE 

DoD  5000. 2-R,  Part  4  establishes  two  funda¬ 
mental  requirements  for  program  management: 

•  It  requires  that  an  Integrated  Product  and 
Process  approach  be  taken  to  design  wherever 
practicable,  and 

•  It  requires  that  a  disciplined  systems  engineer¬ 
ing  process  be  used  to  translate  operational 
needs  and/or  requirements  into  a  system 
solution. 

Tailoring  the  Process 

System  engineering  is  applied  during  all  acquisi¬ 
tion  and  support  phases  for  large-  and  small-scale 
systems,  new  developments  or  product  improve¬ 
ments,  and  single  and  multiple  procurements.  The 
process  must  be  tailored  for  different  needs  and / 
or  requirements.  Tailoring  considerations  include 
system  size  and  complexity,  level  of  system 
definition  detail,  scenarios  and  missions,  con¬ 
straints  and  requirements,  technology  base,  major 
risk  factors,  and  organizational  best  practices  and 
strengths. 

For  example,  systems  engineering  of  software 
should  follow  the  basic  systems  engineering 
approach  as  presented  in  this  book.  However,  it 
must  be  tailored  to  accommodate  the  software 
development  environment,  and  the  unique 


progress  tracking  and  verification  problems  soft¬ 
ware  development  entails.  In  a  like  manner,  all 
technology  domains  are  expected  to  bring  their 
own  unique  needs  to  the  process. 

This  book  provides  a  conceptual-level  description 
of  systems  engineering  management.  The  specific 
techniques,  nomenclature,  and  recommended 
methods  are  not  meant  to  be  prescriptive.  Techni¬ 
cal  managers  must  tailor  their  systems  engineer¬ 
ing  planning  to  meet  their  particular  requirements 
and  constraints,  environment,  technical  domain, 
and  schedule/budget  situation. 

However,  the  basic  time-proven  concepts  inherent 
in  the  systems  engineering  approach  must  be 
retained  to  provide  continuity  and  control.  For 
complex  system  designs,  a  full  and  documented 
understanding  of  what  the  system  must  do  should 
precede  development  of  component  performance 
descriptions,  which  should  precede  component 
detail  descriptions.  Though  some  parts  of  the  sys¬ 
tem  may  be  dictated  as  a  constraint  or  interface,  in 
general,  solving  the  design  problem  should  start 
with  analyzing  the  requirements  and  determining 
what  the  system  has  to  do  before  physical  alterna¬ 
tives  are  chosen.  Configurations  must  be  controlled 
and  risk  must  be  managed. 

Tailoring  of  this  process  has  to  be  done  carefully 
to  avoid  the  introduction  of  substantial  unseen  risk 
and  uncertainty.  Without  the  control,  coordination, 
and  traceability  of  systems  engineering,  an  envi¬ 
ronment  of  uncertainty  results  which  will  lead  to 
surprises.  Experience  has  shown  that  these 
surprises  almost  invariably  lead  to  significant 
impacts  to  cost  and  schedule.  Tailored  processes 
that  reflect  the  general  conceptual  approach  of  this 
book  have  been  developed  and  adopted  by  profes¬ 
sional  societies,  academia,  industry  associations, 
government  agencies,  and  major  companies. 

1.6  SUMMARY  POINTS 

•  Systems  engineering  management  is  a  multi¬ 
functional  process  that  integrates  life  cycle 
functions,  the  systems  engineering  problem 
solving  process,  and  progressive  baselining. 


9 


Systems  Engineering  Fundamentals 


Chapter  1 


•  The  systems  engineering  process  is  a  prob¬ 
lem  solving  process  that  drives  the  balanced 
development  of  system  products  and  processes. 

•  Integrated  Product  Teams  should  apply  the  sys¬ 
tems  engineering  process  to  develop  a  life  cycle 
balanced  design  solution. 

•  The  systems  engineering  process  is  applied  to 
each  level  of  development,  one  level  at  a  time. 

•  Fundamental  systems  engineering  activities  are 
Requirements  Analysis,  Functional  Analysis/ 
Allocation,  and  Design  Synthesis,  all  of  which 
are  balanced  by  System  Analysis  and  Con¬ 
trol. 


•  Baseline  phasing  provides  for  an  increasing 
level  of  descriptive  detail  of  the  products  and 
processes  with  each  application  of  the  systems 
engineering  process. 

•  Baselining  in  a  nut  shell  is  a  concept  descrip¬ 
tion  that  leads  to  a  system  definition  which,  in 
turn,  leads  to  component  definitions,  and  then 
to  component  designs,  which  finally  lead  to 
a  product. 

•  The  output  of  each  application  of  the  systems 
engineering  process  is  a  major  input  to  the  next 
process  application. 
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SYSTEMS  ENGINEERING 
MANAGEMENT 
IN  DOD  ACQUISITION 


2.1  INTRODUCTION 

The  DoD  acquisition  process  has  its  foundation  in 
federal  policy  and  public  law.  The  development, 
acquisition,  and  operation  of  military  systems  is 
governed  by  a  multitude  of  public  laws,  formal 
DoD  directives,  instructions  and  manuals,  numer¬ 
ous  Service  and  Component  regulations,  and  many 
inter-service  and  international  agreements. 

Managing  the  development  and  fielding  of  mili¬ 
tary  systems  requires  three  basic  activities:  tech¬ 
nical  management,  business  management,  and 
contract  management.  As  described  in  this  book, 
systems  engineering  management  is  the  technical 
management  component  of  DoD  acquisition 
management. 

The  acquisition  process  runs  parallel  to  the  require¬ 
ments  generation  process  and  the  budgeting 
process — Planning,  Programming,  and  Budget¬ 
ing  System  (PBBS).  User  requirements  tend  to 
be  event-driven  by  threat.  The  budget  process  is 
date-driven  by  constraints  of  the  Congressional 
calendar.  Systems  Engineering  Management 
bridges  these  processes  and  must  resolve  the  di¬ 
chotomy  of  event-driven  needs,  event-driven 
technology  development,  and  a  calendar  bud¬ 
get. 

Background 

In  1976,  the  Office  of  Management  and  Budget 
(OMB)  published  Circular  A-109  (Major  Systems 
Acquisitions)  with  the  goal  of  increasing  manage¬ 
ment  effectiveness  for  those  acquisitions.  It  laid 
the  foundation  for  standardizing  the  Government 


acquisition  process  and  promoting  unbiased 
concept  definition.  OMB  Circular  A-109  requires 
the  government  agency  to  establish  and  justify  a 
valid  requirement  for  a  capability,  which  must  be 
approved  by  the  executive  agency  head  (Secretary 
of  Defense,  NASA  Administrator,  etc.),  before 
involving  industry  in  the  system  acquisition  pro¬ 
cess.  The  principal  guidance  for  defense  system 
acquisitions  is  the  DoD  5000  series  directives. 
These  documents  reflect  the  actions  required  of 
DoD  acquisition  managers  to: 

•  Translate  operational  needs  into  stable, 
affordable  programs, 

•  Acquire  quality  products,  and 

•  Organize  for  efficiency  and  effectiveness. 

2.2  ACQUISITION  LIFE  CYCLE 

The  acquisition  process  for  major  defense  systems 
is  shown  in  Figure  2-1.  The  process  begins  within 
the  service  or  field  commander-in-chief’s  ongoing 
mission  area  analysis  effort,  which  can  result  in  a 
Mission  Need  Statement  (MNS).  By  certifying  a 
mission  need,  the  MNS  can  result  in  a  decision 
to  explore  material  solutions  to  the  threat  (Mile¬ 
stone  0).  The  program  then  enters  the  Concept 
Exploration  (CE)  phase,  during  which  all  reason¬ 
able  system  alternatives  are  explored.  The  next 
phase  is  Program  Definition  and  Risk  Reduction 
(PDRR).  The  preferred  system  concept  is  defined 
by  a  set  of  system  performance  requirements,  and 
the  technology  is  demonstrated  to  show  that  any 
significant  technical  and  acquisition  risk  areas 
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Figure  2-1 .  Acquisition  Phases 


identified  have  been  brought  under  sufficient 
control  to  warrant  entering  the  next  program  phase. 
The  program  then  enters  the  Engineering  and 
Manufacturing  Development  (EMD)  phase,  where 
the  preliminary  design  is  completed,  detailed 
designs  are  created  and  tests  are  performed,  and 
low-rate  initial  production  is  initiated. 

Following  the  Milestone  (MS)  III  review,  the 
system  enters  the  Production  and  Deployment 
phase,  during  which  full-rate  production  takes 
place.  In  the  Operations  and  Support  phase, 
modifications  and  product  improvements  are 
usually  implemented.  At  the  end  of  the  system 
service  life  it  is  disposed  of  in  accordance  with 
applicable  classified  and  environmental  laws, 
regulations,  and  directives.  Disposal  activities 
also  include  recycling,  material  recovery,  sal¬ 
vage  of  reutilization,  and  disposal  of  by-prod¬ 
ucts  from  development  and  production. 

At  the  end  of  each  of  the  first  three  phases,  the 
need  for  the  program  is  re-certified  by  the  mile¬ 
stone  decision  authority  before  additional  resources 
are  authorized.  At  each  review,  the  decision 
authority  can  choose  to  continue  the  present  phase, 
proceed  to  the  next  phase,  or  cancel  the  program. 


The  decision  authority  may  also  direct  a  tailored 
program  to  omit  or  combine  specific  phases. 
These  special  cases  are  usually  based  on  the  de¬ 
cision  authority  being  convinced  that  the  tech¬ 
nology  and  design  maturity  will  support  such  a 
decision. 


2.3  SYSTEMS  ENGINEERING  IN 
ACQUISITION 

As  required  by  DoD  5000.2-R  (see  Mni-Glos- 

sary),  the  systems  engineering  process  shall: 

1.  Transform  operational  needs  and  requirements 
(reference  Appendix  II)  into  an  integrated 
system  design  solution  through  concurrent 
consideration  of  all  life  cycle  needs  (i.e., 
development,  manufacturing,  test  and  evalua¬ 
tion,  verification,  deployment,  operations, 
support,  training  and  disposal); 

2.  Confirm  the  compatibility,  interoperability  and 
integration  of  all  functional  and  physical  inter¬ 
faces  and  ensure  that  system  definition  and 
design  reflect  the  requirements  for  all  system 
elements:  hardware,  software,  facilities,  people, 
and  data;  and 
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3.  Characterize  and  manage  technical  risks. 

These  objectives  are  accomplished  with  the  use 
of  the  management  concepts  and  techniques 
described  in  the  following  chapters.  The  applica¬ 
tion  of  systems  engineering  management  coincides 
with  acquisition  phasing.  To  support  milestone 
decisions,  major  technical  reviews  are  conducted 
to  evaluate  system  design  maturity. 

Concept  Exploration  (Phase  0) 

As  shown  in  Figure  2-2,  in  Concept  Exploration 
the  primary  inputs  to  the  systems  engineering  pro¬ 
cess  include  the  MNS  and  pre-Milestone  0  out¬ 
puts  developed  by  study  groups.  Alternative  con¬ 
ceptual  solutions  are  developed  during  Concept 
Exploration.  Prior  to  Milestone  I  the  alternative 
concepts  are  reviewed  and  conclusions  concern¬ 
ing  the  technical  approach  for  Phase  I  are  con¬ 


solidated.  The  formal  mechanism  for  this  is  a 
technical  review.  The  output  from  the  systems 
engineering  process  is  used  to  support  the  Mile¬ 
stone  I  decision,  as  well  as  to  provide  informa¬ 
tion  to  help  the  user  develop  the  Operational 
Requirements  Document  (ORD)  and  to  provide 
significant  input  for  the  systems  engineering 
process  in  Phase  I. 

Program  Definition  and 
Risk  Reduction  (Phase  I) 

Major  systems  engineering  inputs  for  PDRR  in¬ 
clude  the  outputs  of  the  process  in  Phase  0,  the 
ORD,  and  the  Phase  I  exit  criteria  established  at 
Milestone  I.  The  systems  engineering  process  will 
be  accomplished  on  various  levels  to  develop  a 
systems  level  (Functional)  baseline  including  a 
System  Specification,  demonstrate  the  technology 
required  to  develop  the  system,  and  identify  and 


13 


Systems  Engineering  Fundamentals 


Chapter  2 


reduce  the  risk  associated  with  developing  the 
chosen  concept(s).  During  this  phase  more  than 
one  version  of  the  basic  concept  may  be  devel¬ 
oped,  usually  in  a  competitive  environment.  As 
shown  in  Figure  2-3,  a  technical  review  is  held  to 
ensure  that  the  phase  objectives  have  been 
achieved.  Technical,  cost,  and  risk  parameters  must 
be  within  acceptable  limits,  and  must  converge  on 
a  complete  and  documented  set  of  system-level 
technical  requirements  (a  System  Specification). 
Systems  engineering  process  outputs  from  PDRR 
become  inputs  for  future  applications  of  the  pro¬ 
cess.  For  example,  the  System  Specification 
approved  in  PDRR  drives  the  preliminary  design 
effort  in  Phase  II. 

The  Milestone  II  decision  to  proceed  to  EMD  is 
highly  dependent  upon  the  quality  of  the  infor¬ 
mation  developed  through  application  of  the  sys¬ 
tems  engineering  process.  That  information  de¬ 
scribes  (or  should)  the  realities  of  the  up-com¬ 
ing  resource  intensive  EMD  and  Production 
phases.  The  technical  information  must  be  cor¬ 


rect  and  complete  and  key  external  interfaces 
identified.  If  the  System  Specification  is  incom¬ 
plete,  the  technology  evaluations  incomplete,  or 
risk  misjudged,  then  the  expectations  reflected 
in  the  Milestone  II  decision  will  not  be  met. 

The  message  is  clear:  PDRR  must  be  used  to 
determine  what  has  to  be  done  in  EMD,  to  develop 
the  technology  to  do  it,  and  to  determine  the  level 
of  difficulty  involved.  Failure  to  fully  consider  the 
technical  realities  at  Milestone  II  will  likely  result 
in  significant  problems  during  EMD.  Technology, 
including  that  necessary  for  integration,  should  be 
developed  in  PDRR  or  pursued  as  a  product 
improvement  effort. 

Technology  development  is  rarely  precisely  pre¬ 
dictable,  but  schedule  and  resources  can  be  planned 
reasonably  close  for  engineering  development. 
EMD  technical  efforts  are  understood  to  be  basi¬ 
cally  engineering  development;  that  is,  they  are  a 
problem  of  consolidating  available  information 
derived  from  past  technology  development. 


Draft  System 
Level  Baseline 


System-Level  Baseline 
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Approved 

System-Level 
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Figure  2-3.  Program  Definition  and  Risk  Reduction  (PDRR) 
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Engineering  and  Manufacturing 
Development  (Phase  II) 

EMD  consists  of  more  than  one  system  engineer¬ 
ing  phase.  It  includes  Preliminary  Design,  Detailed 
Design,  and  preparation  for  the  full-rate  produc¬ 
tion  decision.  These  are  discussed  as  “Stages”  in 
the  paragraphs  that  follow. 

EMD  Stage  1.  Preliminary  Design 

The  first  stage  (Figure  2-4)  is  the  development  of 
the  preliminary  design  based  on  the  system  tech¬ 
nical  requirements  (System  Specification)  devel¬ 
oped  in  Phase  I.  The  systems  engineering  process 
will  be  repeatedly  performed  on  the  subsystem  and 
component  level  to  develop  performance  specifi¬ 
cations  to  describe  the  lower  levels  of  the  system 
architecture.  The  resulting  baseline,  often  referred 
to  as  the  Allocated  Baseline,  consolidates  sub¬ 
system  and  component  technical  performance  and 
interface  requirements.  The  detailed  design  will 
be  developed  from  the  design  requirements 
elaborated  in  the  Allocated  Baseline.  A  techni¬ 


cal  review  is  held  to  ensure  that  the  Allocated 
Baseline  is  complete  and  that  it  will  result  in  an 
appropriate  detailed  design  (Product  Baseline). 

EMD  Stage  2.  Detailed  Design 

Figure  2-5  shows  the  second  stage  of  Phase  II  is 
the  initial  development  of  the  complete  product 
design  in  terms  of  the  physical  components  in¬ 
volved.  This  physical  description  is  the  initial  Prod¬ 
uct  Baseline  definition.  Parts  of  the  Product 
Baseline  are  developed  prior  to  this  period  to  dem¬ 
onstrate  the  validity  of  the  preliminary  design,  or 
because  that  part  was  a  directed  solution,  required 
for  interface,  or  a  non-developmental  item.  The 
baseline  will  continue  to  be  developed  after  this 
period  as  testing  and  initial  production  provide 
information  to  optimize  the  design. 

Final  definition  of  the  baseline  may  not  occur  until 
after  Milestone  III.  However,  the  majority  of  the 
Product  Baseline  is  developed  during  this  period 
through  a  series  of  system  engineering  processes 
focused  on  systems,  subsystems,  and  components. 
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Figure  2-4.  Engineering  and  Manufacturing  Development  (EMD)  -  Stage  1 
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Figure  2-5.  Engineering  and  Manufacturing  Development  (EMD)  -  Stage  2 


When  the  Product  Baseline  is  complete  enough, 
a  review  is  held  to  ensure  that  the  maturity  of 
the  design  is  sufficient  to  begin  initial  low-rate 
production,  initiate  audit  of  the  Allocated  Base¬ 
line,  and  finalize  plans  for  technical  and  opera¬ 
tional  testing.  Parts  of  the  baseline  are  put  under 
formal  control.  This  generally  includes  Item 
Detail  Specifications,  Material  Specifications, 
Process  Specifications,  and  all  drawings  released 
for  production. 

EMD  Stage  3.  Preparation  for  Production 

Variation  often  occurs  between  programs  in  this 
stage.  The  following  describes  a  representative 
approach  to  a  complex,  high-rate  production  sytem. 
It  includes  the  basic  activities  and  sequences  in¬ 
herent  in  this  stage.  Shown  by  Figure  2-6,  the  third 
stage  of  EMD  consists  of  continued  detail  design, 
system  verification,  and  initial  production.  The 
three  activities  run  in  parallel  and  the  success  of 
each  depends  on  the  others.  Design  refinement  will 
depend  on  feedback  from  testing  and  produc¬ 
tion. 


Testing  will  be  more  meaningful  if  the  systems 
engineering  efforts  result  in  a  testable  system 
configuration  that  meets  customer  expectations. 
Initial  production  will  go  more  smoothly  if  the 
system  configuration  is  designed  to  be  produc¬ 
ible.  Well-produced  initial  production  units  will 
help  ensure  that  the  tests  and  design  audits  are 
successful. 

Continued  Design  Effort  -  The  Product  Baseline 
continues  to  be  developed  to  greater  detail  with 
input  from  system  engineering  process  verifica¬ 
tion  and  analysis  efforts,  formal  testing,  and  ini¬ 
tial  production.  Audits  of  system  components  are 
held  to  verify  they  meet  their  Allocated  Baseline 
requirements.  After  all  appropriate  components 
have  been  audited,  a  technical  review  is  held  to 
confirm  that  the  Allocated  Baseline  matches  the 
as-built  component  configurations;  and  based  on 
evaluating  production  representative  prototypes  or 
early  production  units  that  the  as-built  system 
configuration  matches  the  Functional  Baseline. 
The  findings  of  this  review  are  major  inputs  to 
the  Milestone  III  decision  process. 
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Figure  2-6.  Engineering  and  Manufacturing  Development  (EMD)  -  Stage  3 


Initial  Production  -  After  the  technical  review 
in  the  previous  stage  confirms  the  configuration 
is  suitable  to  initiate  limited  production,  a  series 
of  readiness  reviews  are  held  to  confirm  the  pro¬ 
duction  process  is  ready  and  in  place.  Low  Rate 
Initial  Production  (LRIP)  is  then  initiated  to  sup¬ 
port  testing,  provide  feedback  to  design  efforts, 
gradually  ramp-up  to  rate-production  levels,  and 
develop  the  production  process  in  an  orderly 
manner.  Early  production  units  can  be  used  to 
support  Operational  Testing.  (Though  it  is  not 
unusual  for  some  system  level  developmental 
testing  to  be  done  on  early  production  units, 
developmental  testing  does  not  satisfy  the 
statutory  requirements  to  justify  production 
of  low-rate  items  prior  to  the  Milestone  III  de¬ 
cision.)  The  effectiveness  and  stability  of  the 
initial  production  process  is  a  major  consider¬ 
ation  in  the  Milestone  III  decision  process. 


Test  and  Evaluation  -  After  the  technical  re¬ 
view  in  the  previous  stage,  readiness  reviews  are 
held  to  confirm  testing  processes  are  ready  and  in 
place.  Live  fire  and  developmental  testing  is  per¬ 
formed  by  the  developing  agency  to  support  the 
design  process  and  to  prepare  for  independent 
operational  testing.  Initial  Operational  Test  and 
Evaluation  (IOT&E)  is  then  performed  on  a  pro¬ 
duction  representative  system  by  the  service 
operational  test  agency.  This  test  assesses 
whether  the  operational  requirements  have  been 
met  and  if  the  system  is  operationally  effective 
and  suitable.  Test  results  are  used  to  refine  the 
design.  The  operational  test  report  is  a  major 
(usually  the  most  important)  input  to  the  Mile¬ 
stone  III  decision  process.  The  results  of  the  design, 
test,  and  initial  production  processes  are  the  tech¬ 
nical  inputs  to  the  full-rate  production  decision. 
From  audits  and  testing,  design  maturity  and  risk 
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Figure  2-7.  Production  and  Deployment 


is  assessed.  Initial  production  confirms  production 
capability  and  unit  cost  affordability. 

Production,  Deployment,  Operation,  and 
Support  (Phase  III) 

After  the  decision  to  go  to  full-rate  production, 
the  systems  engineering  process  is  used  to  re¬ 
fine  the  detail  design  to  incorporate  findings  of 
the  independent  operational  testing,  direction 
from  the  milestone  decision  authority,  and  feed¬ 
back  from  deployment  activities.  Once  configu¬ 
ration  changes  have  been  made  and  incorporated 
into  production,  and  the  configuration  and  pro¬ 
duction  is  considered  stable,  Follow-on  Opera¬ 
tional  Test  and  Evaluation  (FOT&E)  is  performed 
on  a  stable  production  system.  Test  results  are 
used  to  further  refine  the  production  configura¬ 


tion.  Once  this  has  been  accomplished  and  pro¬ 
duction  again  becomes  stable,  a  series  of  de¬ 
tailed  audits  are  held  to  confirm  that  the  Product 
Baseline  matches  the  system  being  produced. 
Once  the  audits  and  any  resulting  corrections 
have  been  successfully  completed,  the  Product 
Baseline  is  put  under  formal  configuration  con¬ 
trol. 

The  configuration  is  then  formally  managed  for 
the  life  of  the  system,  usually  at  all  three  baseline 
levels.  Systems  Engineering  activities  in  the  op¬ 
eration  and  support  phase  are  focused  on  control¬ 
ling  the  configuration,  especially  maintaining  the 
system’s  performance  capability.  If  the  military 
threat  changes  or  a  technology  opportunity 
emerges,  then  the  system  may  require  a  modifica¬ 
tion.  These  modifications  must  be  approved  at  an 
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appropriate  level  for  the  particular  change  being 
considered.  The  change  then  drives  the  initia¬ 
tion  of  new  systems  engineering  processes,  start¬ 
ing  the  cycle  (or  parts  of  it)  over  again. 

Disposal 

System  engineers  plan  for,  and  conduct,  system 
disposal  throughout  the  life  cycle,  beginning  with 
concept  development.  System  components  can 
require  disposal  because  of  decommissioning,  their 
destruction,  or  irreparable  damage.  In  addition, 
processes  and  material  used  for  development, 
production,  operation,  or  maintenance  can  raise 
disposal  issues  throughout  the  life  cycle. 

Disposal  must  be  done  in  accordance  with  appli¬ 
cable  and  laws,  regulations,  and  directives  that  are 
continually  changing,  usually  to  require  more 
severe  constraints.  They  mostly  relate  to  security 
and  environment  issues  that  include  recycling, 
material  recovery,  salvage,  and  disposal  of  by¬ 
products  from  development  and  production. 

Every  Development  Is  Different 

The  process  outlined  above  is  the  “ideal”  or  “nomi¬ 
nal”  development  that  would  normally  apply  to  a 
major  acquisition.  The  systems  engineer  has  to 
tailor  this  nominal  process  to  the  specific  devel¬ 
opment.  For  example,  if  the  system  design  will 
rely  significantly  on  the  use  of  commercial  items, 
then  the  product’s  detailed  design  and  fabrication 
can  be  adjusted  to  a  more  appropriate,  low-level 
effort.  If  the  type  of  system  is  well  understood 
within  the  applicable  technical  domains,  or  if  it 
is  an  advanced  version  of  a  current,  well  under¬ 
stood  system,  then  the  program  definition  and 
risk  reduction  efforts  could  be  adjusted  to  a  lower- 
level  effort. 


The  process  must  be  tailored  to  the  specific  de¬ 
velopment,  both  because  it  is  good  engineering 
and  because  it  is  DoD  policy  as  part  of  the  Ac¬ 
quisition  Reform  initiative.  But  tailoring  must 
be  done  with  the  intent  of  preserving  the  require¬ 
ments  traceability,  baseline  control,  life  cycle 
focus,  maturity  tracking,  and  integration  inher¬ 
ent  in  the  systems  engineering  approach.  The 
validity  of  tailoring  the  process  should  always 
be  a  risk  management  issue.  Acquisition  Reform 
issues  will  be  addressed  again  in  Part  IV. 


2.4  SUMMARY  POINTS 

•  The  development,  acquisition,  and  operation  of 
military  systems  is  governed  by  a  multitude  of 
public  laws,  formal  DoD  directives,  instructions 
and  manuals,  numerous  Service  and  Compo¬ 
nent  regulations,  and  many  inter-service  and 
international  agreements. 

•  Systems  engineering  management  must  resolve 
the  dichotomy  of  threat-driven  needs,  event- 
driven  technology  development,  and  a  calendar 
budget. 

•  The  consequence  of  an  incomplete  PDRR  sys¬ 
tems  engineering  effort  is  that  the  expectations 
of  the  decision  makers  formed  at  Milestone 
II  will  not  be  met. 

•  Systems  engineering  management  is  a  critical 
support  process  for  DoD  Acquisition.  If  the 
systems  engineering  management  is  success¬ 
ful,  then  the  program  will  likely  be  successful. 
If  the  systems  engineering  management  effort 
fails,  then  the  Program  Acquisition  effort  will 
also  fail. 

•  Finally,  Figure  2-8  provides  an  overview  of  how 
systems  engineering  is  performed  throughout 
the  acquisition  life  cycle. 
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Figure  2-8.  Systems  Engineering  and  the  Acquisition  Life  Cycle 
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SYSTEMS  ENGINEERING 
PROCESS  OVERVIEW 


3.1  THE  PROCESS 

The  Systems  Engineering  Process  (SEP)  is  a  com¬ 
prehensive,  iterative  &  recursive  problem  solving 
process,  applied  sequentially  top-down  by  inte¬ 
grated  teams.  It  transforms  needs  and  requirements 
into  a  set  of  system  product  and  process  descrip¬ 
tions,  generate  information  for  decision-makers, 
and  provides  input  for  the  next  level  of  develop¬ 
ment.  The  process  is  applied  sequentially,  one  level 
at  a  time,  adding  additional  detail  and  definition 


with  each  level  of  development.  As  shown  by 
Figure  3-1,  the  process  includes:  inputs  and  out¬ 
puts;  requirements  analysis;  functional  analy¬ 
sis  and  allocation;  requirements  loop;  synthesis; 
design  loop;  verification;  and  system  analysis 
and  control. 

SE  Process  Inputs 

Inputs  consist  primarily  of  the  customer’s  needs, 
objectives,  requirements  and  project  constraints. 


Process  Input 

•  Customer  Needs/Objectives/ 
Requirements 

-  Missions 

-  Measures  of  Effectiveness 

-  Environments 

-  Constraints 

•  Technology  Base 

•  Output  Requirements  from  Prior 
Development  Effort 

•  Program  Decision  Requirements 

•  Requirements  Applied  Through  / 
Specifications  and  Standards  / 


System  Analysis 
&  Control 

.  (Balance)  > 


Requirements  Analysis 

Analyze  Missions  &  Environments 
Identify  Functional  Requirements 
Define/Refine  Performance  &  Design 
Constraint  Requirements 

Requirements  Loop 


Functional  Analysis/Allocation 

Decompose  to  Lower-Level  Functions 

Allocate  Performance  &  Other  Limiting  Requirements  to  - 

All  Functional  Levels 

Define/Refine  Functional  Interfaces  (Internai/External) 
Define/Refine/Integrate  Functional  Architecture 


Design  Loop  _ 

Synthesis 

Transform  Architectures  (Functional  to  Physical) 
Define  Alternative  System  Concepts,  Configuration 
Items  &  System  Elements 
Select  Preferred  Product  &  Process  Solutions 
Define/Refine  Physical  Interfaces  (Internal/Externa!) 


Verification 


Related  Terms: 

Customer  = 
Primary  Functions  = 

Systems  Elements  = 


Organizations  responsible  for  Primary  Functions 
Development,  Production/Construction,  Verification, 
Deployment,  Operations,  Support,  Training,  Disposal 
Hardware,  Software,  Personnel,  Facilities,  Data,  Material, 
Services,  Techniques 


Trade-Off  Studies  ' 

Effectiveness  Analyses 
Risk  Management 
Configuration  Management 
Interface  Management 
Data  Management 
Performance  Measurement 
-  SEMS 
-TPM 

-Technical  Reviews 


Process  Output 

•  Development  Level  Dependent 

-  Decision  Data  Base 

-  System/Configuration  Item 
Architecture 

-  Specifications  &  Baselines 


Figure  3-1.  The  Systems  Engineering  Process 
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Inputs  can  include,  but  are  not  restricted  to,  mis¬ 
sions,  measures  of  effectiveness,  environments, 
available  technology  base,  output  requirements 
from  prior  application  of  the  systems  engineering 
process,  program  decision  requirements,  and 
requirements  based  on  “corporate  knowledge.” 

Requirements  Analysis 

The  first  step  of  the  Systems  Engineering  Process 
is  to  analyze  the  process  inputs.  Requirements  analy¬ 
sis  is  used  to  develop  functional  and  performance 
requirements;  that  is,  customer  requirements  are 
translated  into  a  set  of  requirements  that  define 
what  the  system  must  do  and  how  well  it  must 
perform.  The  systems  engineer  must  ensure  that  the 
requirements  are  understandable,  unambiguous, 
comprehensive,  complete,  and  concise. 

Requirements  analysis  must  clarify  and  define 
functional  requirements  and  design  constraints. 
Functional  requirements  define  quantity  (how 
many),  quality  (how  good),  coverage  (how  far),  time 
lines  (when  and  how  long),  and  availability  (how 
often).  Design  constraints  define  those  factors 
that  limit  design  flexibility,  such  as: 
environmental  conditions  or  limits;  defense  against 
internal  or  external  threats;  and  contract,  customer 
or  regulatory  standards. 

Functional  Analysis/Allocation 

Functions  are  analyzed  by  decomposing  higher- 
level  functions  identified  through  requirements 
analysis  into  lower-level  functions.  The  perfor¬ 
mance  requirements  associated  with  the  higher 
level  are  allocated  to  lower  functions.  The  result 
is  a  description  of  the  product  or  item  in  terms  of 
what  it  does  logically  and  in  terms  of  the  perfor¬ 
mance  required.  This  description  is  often  called 
the  functional  architecture  of  the  product  or  item. 
Functional  analysis  and  allocation  allows  for  a 
better  understanding  of  what  the  system  has  to  do, 
in  what  ways  it  can  do  it,  and  to  some  extent,  the 
priorities  and  conflicts  associated  with  lower-level 
functions.  It  provides  information  essential  to 
optimizing  physical  solutions.  Key  tools  in  func¬ 
tional  analysis  and  allocation  are  Functional  Flow 
Block  Diagrams,  Time  Line  Analysis,  and  the 
Requirements  Allocation  Sheet. 


Requirements  Loop 

Performance  of  the  functional  analysis  and  allo¬ 
cation  results  in  a  better  understanding  of  the 
requirements  and  should  prompt  reconsideration 
of  the  requirements  analysis.  Each  function  iden¬ 
tified  should  be  traceable  back  to  a  requirement. 
This  iterative  process  of  revisiting  requirements 
analysis  as  a  result  of  functional  analysis  and 
allocation  is  referred  to  as  the  requirements  loop. 

Design  Synthesis 

Design  synthesis  is  the  process  of  defining  the 
product  or  item  in  terms  of  the  physical  and  soft¬ 
ware  elements  which  together  make  up  and  define 
the  item.  The  result  is  often  referred  to  as  the  physi¬ 
cal  architecture.  Each  part  must  meet  at  least  one 
functional  requirement,  and  any  part  may  support 
many  functions.  The  physical  architecture  is  the 
basic  structure  for  generating  the  specifications  and 
baselines. 

Design  Loop 

Similar  to  the  requirements  loop  described  above, 
the  design  loop  is  the  process  of  revisiting  the  func¬ 
tional  architecture  to  verify  that  the  physical  design 
synthesized  can  perform  the  required  functions  at 
required  levels  of  performance.  The  design  loop 
permits  reconsideration  of  how  the  system  will 
perform  its  mission,  and  this  helps  optimize  the 
synthesized  design. 

Verification 

For  each  application  of  the  system  engineering 
process,  the  solution  will  be  compared  to  the 
requirements.  This  part  of  the  process  is  called  the 
verification  loop,  or  more  commonly.  Verification. 
Each  requirement  at  each  level  of  development 
must  be  verifiable.  Baseline  documentation  devel¬ 
oped  during  the  systems  engineering  process  must 
establish  the  method  of  verification  for  each 
requirement. 

Appropriate  methods  of  verification  include 
examination,  demonstration,  analysis  (including 
modeling  and  simulation),  and  testing.  Formal 
test  and  evaluation  (both  developmental  and 
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operational)  are  important  contributors  to  the 
verification  of  systems. 

Systems  Analysis  and  Control 

Systems  Analysis  and  Control  include  technical 
management  activities  required  to  measure 
progress,  evaluate  and  select  alternatives,  and  docu¬ 
ment  data  and  decisions.  These  activities  apply  to 
all  steps  of  the  SE  process. 

System  analysis  activities  include  trade-off  stud¬ 
ies,  effectiveness  analyses,  and  design  analyses. 
They  evaluate  alternative  approaches  to  satisfy 
technical  requirements  and  program  objectives, 
and  provide  a  rigorous  quantitative  basis  for 
selecting  performance,  functional,  and  design 
requirements.  Tools  used  to  provide  input  to 
analysis  activities  include  modeling,  simulation, 
experimentation,  and  test. 

Control  activities  include  risk  management,  con¬ 
figuration  management,  data  management,  and 
performance-based  progress  measurement  includ¬ 
ing  event  based  scheduling,  Technical  Performance 
Measurement  (TPM),  and  technical  reviews. 

The  purpose  of  Systems  Analysis  and  Control  is 
to  ensure  that: 

•  Solution  alternative  decisions  are  made  only 
after  evaluating  the  impact  on  system  effective¬ 
ness,  life  cycle  resources,  risk,  and  customer 
requirements; 


•  Technical  decisions  and  specification  require¬ 
ments  are  based  on  systems  engineering 
outputs; 

•  Traceability  from  systems  engineering  process 
inputs  to  outputs  is  maintained; 

•  Schedules  for  development  and  delivery  are 
mutually  supportive; 

•  Required  technical  disciplines  are  integrated 
into  the  systems  engineering  effort; 

•  Impacts  of  customer  requirements  on  resulting 
functional  and  performance  requirements  are 
examined  for  validity,  consistency,  desirabil¬ 
ity,  and  attainability;  and, 

•  Product  and  process  design  requirements  are 
directly  traceable  to  the  functional  and  perfor¬ 
mance  requirements  they  were  designed  to 
fulfill,  and  vice  versa. 

SE  Process  Output 

Process  output  is  dependent  on  the  level  of  devel¬ 
opment.  It  will  include  the  decision  database,  the 
system  or  configuration  item  architecture,  and  the 
baselines,  including  specifications,  appropriate  to 
the  phase  of  development.  In  general,  it  is  any  data 
that  describes  or  controls  the  product  configura¬ 
tion  or  the  processes  necessary  to  develop  that 
product. 
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REQUIREMENTS 

ANALYSIS 


4.1  SYSTEMS  ENGINEERING  PROCESS 
INPUTS 

The  inputs  to  the  process  include  the  customer’s 
requirements  and  the  project  constraints.  Require¬ 
ments  relate  directly  to  the  performance  charac¬ 
teristics  of  the  system  being  designed.  They  are 
the  stated  life-cycle  customer  needs  and  objec¬ 
tives  for  the  system,  and  they  relate  to  how  well 
the  system  will  work  in  its  intended  environ¬ 
ment. 

Constraints  are  conditions  that  exist  because  of 
limitations  imposed  by  external  interfaces,  project 
support,  technology,  or  life  cycle  support  systems. 
Constraints  bound  the  development  teams’  design 
opportunities. 

Requirements  are  the  primary  focus  in  the  sys¬ 
tems  engineering  process  because  the  process’s 
primary  purpose  is  to  transform  the  requirements 


into  designs.  The  process  develops  these  designs 
within  the  constraints.  They  eventually  must  be 
verified  to  meet  both  the  requirements  and 
constraints. 

Types  of  Requirements 

Requirements  are  categorized  in  several  ways.  The 
following  are  common  categorizations  of  require¬ 
ments  that  relate  to  technical  management: 

Customer  Requirements:  Statements  of  fact  and 
assumptions  that  define  the  expectations  of  the 
system  in  terms  of  mission  objectives,  environ¬ 
ment,  constraints,  and  measures  of  effectiveness 
and  suitability  (MOE/MOS).  The  customers  are 
those  that  perform  the  eight  primary  functions  of 
systems  engineering  (Chapter  1),  with  special 
emphasis  on  the  operator  as  the  key  customer. 
Operational  requirements  will  define  the  basic  need 
and,  at  a  minimum,  answer  the  questions  posed  in 
Figure  4-1. 


Operational  distribution  or  deployment:  Where  will  the  system  be  used? 

Mission  profile  or  scenario:  How  will  the  system  accomplish  its  mission  objective? 

Performance  and  related  parameters:  What  are  the  critical  system  parameters  to  accom¬ 
plish  the  mission? 

Utilization  environments:  How  are  the  various  system  components  to  be  used? 

Effectiveness  requirements:  How  effective  or  efficient  must  the  system  be  in  performing  its 
mission? 

Operational  life  cycle:  How  long  will  the  system  be  in  use  by  the  user? 

Environment:  In  what  environments  will  the  system  be  expected  to  operate  (in  an  effective 
manner)? 

Figure  4-1 .  Operational  Requirements  -  Basic  Questions 
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Functional  Requirements:  The  necessary  task, 
action  or  activity  that  must  be  accomplished. 
Functional  (what  has  to  be  done)  requirements 
identified  in  requirements  analysis  will  be  used 
as  the  top-level  functions  for  functional  analy¬ 
sis. 

Performance  Requirements:  The  extent  to  which 
a  mission  or  function  must  be  executed;  generally 
measured  in  terms  of  quantity,  quality,  coverage, 
timeliness  or  readiness.  During  requirements 
analysis,  performance  (how  well  does  it  have  to 
be  done)  requirements  will  be  interactively  devel¬ 
oped  across  all  identified  functions  based  on  sys¬ 
tem  life  cycle  factors;  and  characterized  in  terms 
of  the  degree  of  certainty  in  their  estimate,  the 
degree  of  criticality  to  system  success,  and  their 
relationship  to  other  requirements. 

Design  Requirements:  The  “build  to,”  “code  to,” 
and  “buy  to”  requirements  for  products  and  “how 
to  execute”  requirements  for  processes  expressed 
in  technical  data  packages  and  technical  manuals. 

Derived  Requirements:  Requirements  that  are 
implied  or  transformed  from  higher-level  require¬ 
ment.  For  example,  a  requirement  for  long  range 
or  high  speed  may  result  in  a  design  requirement 
for  low  weight. 

Allocated  Requirements:  A  requirement  that  is 
established  by  dividing  or  otherwise  allocating 
a  high-level  requirement  into  multiple  lower- 
level  requirements.  Example:  A  100-pound  item 
that  consists  of  two  subsystems  might  result  in 
weight  requirements  of  70  pounds  and  30  pounds 
for  the  two  lower-level  items. 

Attributes  of  Good  Requirement 

The  attributes  of  good  requirements  include  the 
following: 

•  A  requirement  must  be  achievable.  It  must 
reflect  need  or  objective  for  which  a  solution  is 
technically  achievable  at  costs  considered 
affordable. 


•  It  must  be  verifiable — that  is,  not  defined  by 
words  such  as  excessive,  sufficient,  resistant, 
etc.  The  expected  performance  and  functional 
utility  must  be  expressed  in  a  manner  that 
allows  verification  to  be  objective,  preferably 
quantitative. 

•  A  requirement  must  be  unambiguous.  It  must 
have  but  one  possible  meaning. 

•  It  must  be  complete  and  contain  all  mission 
profiles,  operational  and  maintenance  concepts, 
utilization  environments  and  constraints.  All 
information  necessary  to  understand  the 
customer’s  need  must  be  there. 

•  It  must  be  expressed  in  terms  of  need,  not 
solution;  that  is,  it  should  address  the  “why” 
and  “what”  of  the  need,  not  how  to  do  it. 

•  It  must  be  consistent  with  other  requirements. 
Conflicts  must  be  resolved  up  front. 

•  It  must  be  appropriate  for  the  level  of  system 
hierarchy.  It  should  not  be  too  detailed  that  it 
constrains  solutions  for  the  current  level  of 
design.  For  example,  detailed  requirements 
relating  to  components  would  not  normally  be 
in  a  system-level  specification. 

4.2  REQUIREMENTS  ANALYSIS 

Requirements  analysis  involves  defining  customer 
needs  and  objectives  in  the  context  of  planned 
customer  use,  environments,  and  identified  system 
characteristics  to  determine  requirements  for 
system  functions.  Prior  analyses  are  reviewed  and 
updated,  refining  mission  and  environment 
definitions  to  support  system  definition. 

Requirements  analysis  is  conducted  iteratively  with 
functional  analysis  to  optimize  performance 
requirements  for  identified  functions,  and  to  verify 
that  synthesized  solutions  can  satisfy  customer 
requirements.  The  purpose  of  Requirements 
Analysis  is  to: 
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•  Refine  customer  objectives  and  requirements;  Inputs 


•  Define  initial  performance  objectives  and  Typical  inputs  include  customer  needs  and  objec- 

refine  them  into  requirements;  tives,  missions,  MOE/MOS,  environments,  key 

performance  parameters  (KPPs),  technology  base, 

•  Identify  and  define  constraints  that  limit  output  requirements  from  prior  application  of  SEP, 

solutions;  and  program  decision  requirements,  and  suitability 

requirements.  (See  Figure  4-2  for  additional 

•  Define  functional  and  performance  require-  considerations.) 
ments  based  on  customer  provided  measures 

of  effectiveness.  Input  requirements  must  be  comprehensive  and 

defined  for  both  system  products  and  system 
In  general,  Requirements  Analysis  should  result  processes  such  as  development,  manufacturing, 
in  a  clear  understanding  of:  verification,  deployment,  operations,  support, 

training  and  disposal  (eight  primary  functions). 

•  Functions:  What  the  system  has  to  do, 

Role  of  Integrated  Teams 

•  Performance:  How  well  the  functions  have  to 

be  performed,  The  operator  customers  have  expertise  in  the 

operational  employment  of  the  product  or  item 

•  Interfaces:  Environment  in  which  the  system  being  developed.  The  developers  (government  and 

will  perform,  and  contractor)  are  not  necessarily  competent  in  the 

operational  aspects  of  the  system  under  develop- 

•  Other  requirements  and  constraints.  ment.  Typically,  the  operator’s  need  is  neither 

clearly  nor  completely  expressed  in  a  way  directly 
The  understandings  that  come  from  requirements  usable  by  developers.  It  is  unlikely  that  develop- 
analysis  establish  the  basis  for  the  functional  and  ers  will  receive  a  well-defined  problem  from 
physical  designs  to  follow.  Good  requirements 
analysis  is  fundamental  to  successful  design 
definition. 


•  Inputs  converted  to  outputs: 

-  Customer  requirements 

-  Mission  and  MOEs  (MNS,  ORD) 

-  Maintenance  concept  and  other  life-cycle  function 
planning 

-  SE  outputs  from  prior  development  efforts 

Controls 

•  Controls: 

-  Laws  and  organizational  policies  and  procedures 

4 

-  Military  specific  requirements 

-  Utilization  environments  inputs 

-  Tech  base  and  other  constraints  Transformed 

into  Outputs  ^ 

•  Enablers: 

_  Mi  ilti-rliQrinlinarv  rirnHiiPt  tpamc; 

Requirement* 

Analysis 

Outputs 

-  IVIUlU  UloGipJiii  icti  y  pi  uuuut  icrciiiio 

-  Decision  and  requirements  database  including 
system/configuration  item  descriptions  from  prior 
efforts 

-  System  analysis  and  control 

t 

Enablers 

Figure  4-2.  Inputs  to  Requirements  Analysis 
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which  they  can  develop  the  system  specifica¬ 
tion.  Thus,  teamwork  is  necessary  to 
understand  the  problem  and  to  analyze  the  need. 
It  is  imperative  that  customers  are  part  of  the 
definition  team. 

On  the  other  hand,  customers  often  find  it  easier 
to  describe  a  system  that  attempts  to  solve  the  prob¬ 
lem  rather  than  to  describe  the  problem  itself. 
Although  these  “solutions”  may  be  workable  to 
some  extent,  the  optimum  solution  is  obtained 
through  a  proper  technical  development  effort 
that  properly  balances  the  various  customer 
mission  objectives,  functions,  MOE/MOS,  and 
constraints.  An  integrated  approach  to  product 
and  process  development  will  balance  the  analy¬ 
sis  of  requirements  by  providing  understanding 
and  accommodation  among  the  eight  primary 
functions. 

Requirements  Analysis  Questions 

Requirements  Analysis  is  a  process  of  inquiry  and 
resolution.  The  following  are  typical  questions  that 
can  initiate  the  thought  process: 

•  What  are  the  reasons  behind  the  system 
development? 

•  What  are  the  customer  expectations? 

•  Who  are  the  users  and  how  do  they  intend  to 
use  the  product? 

•  What  do  the  users  expect  of  the  product? 

•  What  is  their  level  of  expertise? 

•  With  what  environmental  characteristics  must 
the  system  comply? 

•  What  are  existing  and  planned  interfaces? 

•  What  functions  will  the  system  perform, 
expressed  in  customer  language? 

•  What  are  the  constraints  (hardware,  software, 
economic,  procedural)  to  which  the  system 
must  comply? 


•  What  will  be  the  final  form  of  the  product: 
such  as  model,  prototype,  or  mass  produc¬ 
tion? 

This  list  can  start  the  critical,  inquisitive  outlook 
necessary  to  analyze  requirements,  but  it  is  only 
the  beginning.  A  tailored  process  similar  to  the 
one  at  the  end  of  this  chapter  must  be  developed 
to  produce  the  necessary  requirements  analysis 
outputs. 

4.3  REQUIREMENTS  ANALYSIS 
OUTPUTS 

The  requirements  that  result  from  requirements 
analysis  are  typically  expressed  from  one  of  three 
perspectives  or  views.  These  have  been  described 
as  the  Operational,  Functional,  and  Physical  views. 
All  three  are  necessary  and  must  be  coordinated 
to  fully  understand  the  customers’  needs  and 
objectives.  All  three  are  documented  in  the  decision 
database. 

Operational  View 

The  Operational  View  addresses  how  the  system 
will  serve  its  users.  It  is  useful  when  establishing 
requirements  of  “how  well”  and  “under  what  con¬ 
dition.”  Operational  view  information  should  be 
documented  in  an  operational  concept  document 
that  identifies: 

•  Operational  need  definition, 

•  System  mission  analysis, 

•  Operational  sequences, 

•  Operational  environments, 

•  Conditions/events  to  which  a  system  must 
respond, 

•  Operational  constraints  on  system, 

•  Mission  performance  requirements, 
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•  User  and  maintainer  roles  (defined  by  job  tasks 
and  skill  requirements  or  constraints), 

•  Structure  of  the  organizations  that  will  operate, 
support  and  maintain  the  system,  and 

•  Operational  interfaces  with  other  systems. 

Analyzing  requirements  requires  understanding 
the  operational  and  other  life  cycle  needs  and 
constraints. 

Functional  View 

The  Functional  View  focuses  on  WHAT  the  sys¬ 
tem  must  do  to  produce  the  required  operational 
behavior.  It  includes  required  inputs,  outputs, 
states,  and  transformation  rules.  The  functional 
requirements,  in  combination  with  the  physical 
requirements  shown  below,  are  the  primary 
sources  of  the  requirements  that  will  eventually 
be  reflected  in  the  system  specification.  Func¬ 
tional  View  information  includes: 

•  System  functions, 

•  System  performance, 

-  Qualitative  —  how  well 

-  Quantitative  —  how  much,  capacity 

-  Timeliness  or  periodicity  —  how  often 

•  Tasks  or  actions  to  be  performed, 

•  Inter-function  relationships, 

•  Hardware  and  software  functional  relationships, 

•  Performance  constraints, 

•  Interface  requirements  including  identification 
of  potential  open-system  opportunities  (po¬ 
tential  standards  that  could  promote  open  sys¬ 
tems  should  be  identified), 

•  Unique  hardware  or  software,  and 

•  Verification  requirements  (to  include  inspec¬ 
tion,  analysis/simulation,  demo,  and  test). 


Physical  View 

The  Physical  View  focuses  on  HOW  the  system 
is  constructed.  It  is  key  to  establishing  the  physi¬ 
cal  interfaces  among  operators  and  equipment, 
and  technology  requirements.  Physical  View 
information  would  normally  include: 

•  Configuration  of  System: 

-  Interface  descriptions, 

-  Characteristics  of  information  displays  and 
operator  controls, 

-  Relationships  of  operators  to  system/ 
physical  equipment,  and 

-  Operator  skills  and  levels  required  to 
perform  assigned  functions. 

•  Characterization  of  Users: 

-  Handicaps  (special  operating  environments), 
and 

-  Constraints  (movement  or  visual  limita¬ 
tions.) 

•  System  Physical  Limitations: 

-  Physical  limitations  (capacity,  power,  size, 
weight), 

-  Technology  limitations  (range,  precision, 
data  rates,  frequency,  language), 

-  Government  Furinished  Equipment  (GFE), 
Commercial-Off-the-Shelf  (COTS), 
Nondevelopmental  Item  (NDI),  reusabil¬ 
ity  requirements,  and 

—  Necessary  or  directed  standards. 

4.4  SUMMARY  POINTS 

•  An  initial  statement  of  a  need  is  seldom  defined 
clearly. 

•  A  significant  amount  of  collaboration  between 
various  life  cycle  customers  is  necessary  to 
produce  an  acceptable  requirements  document. 

•  Requirements  are  a  statement  of  the  problem 
to  be  solved.  Unconstrained  and  noninte- 
grated  requirements  are  seldom  sufficient  for 
designing  a  solution. 
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•  Because  requirements  from  different  custom-  must  be  accomplished  in  order  to  select  a  bal¬ 
ers  will  conflict,  constraints  will  limit  options,  anced  set  of  requirements  that  provide  fea- 

and  resources  are  not  unlimited;  trade  studies  sible  solutions  to  customer  needs. 
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SUPPLEMENT  A 

A  PROCEDURE  FOR  REQUIREMENTS 

ANALYSIS 


The  following  section  provides  a  list  of  tasks 
that  represents  a  plan  to  analyze  requirements. 
Part  of  this  notional  process  is  based  on  the  15 
requirements  analysis  tasks  listed  in  IEEE  P1220. 
This  industry  standard  and  others  should  be  con¬ 
sulted  when  preparing  engineering  activities  to 
help  identify  and  structure  appropriate  activities. 

As  with  all  techniques,  the  student  should  be 
careful  to  tailor;  that  is,  add  or  subtract,  as  suits 
the  particular  system  being  developed.  Addition¬ 
ally,  these  tasks,  though  they  build  on  each  other, 
should  not  be  considered  purely  sequential.  Ev¬ 
ery  task  contributes  understanding  that  may  cause 
a  need  to  revisit  previous  task  decisions.  This  is 
the  nature  of  all  System  Engineering  activities. 

Preparation:  Establish  and 
Maintain  Decision  Database 

When  beginning  a  systems  engineering  process, 
be  sure  that  a  system  is  in  place  to  record  and  man¬ 
age  the  decision  database.  The  decision  database 


is  a  historical  database  of  technical  decisions  and 
requirements  for  future  reference.  It  is  the  pri¬ 
mary  means  for  maintaining  requirements  trace- 
ability.  This  database  decision  management  sys¬ 
tem  must  be  developed  or  the  existing  system  must 
be  reviewed  and  upgraded  as  necessary  to  accom¬ 
modate  the  new  stage  of  product  development.  A 
key  part  of  this  database  management  system  is 
a  Requirements  Traceability  Matrix  that  maps 
requirements  to  subsystems,  configuration  items, 
and  functional  areas. 

This  must  be  developed,  updated,  and  reissued  on 
a  regular  basis.  All  requirements  must  be  recorded. 
Remember:  If  it  is  not  recorded,  it  cannot  be 
an  approved  requirement! 

The  Fifteen  Tasks  of  IEEE  P1220 

The  IEEE  Systems  Engineering  Standard  offers  a 
process  for  performing  Requirements  Analysis  that 
comprehensively  identifies  the  important  tasks  that 
must  be  performed.  These  fifteen  task  areas  to  be 
analyzed  follow  and  are  shown  in  Figure  4-3. 


1.  Customer  expectations 

2.  Project  and  enterprise  constraints 

3.  External  constraints 

4.  Operational  scenarios 

5.  Measures  of  Effectiveness  (MOEs) 

6.  System  boundaries 

7.  Interfaces 

8.  Utilization  environments 


9.  Life  cycle  process  concepts 

10.  Functional  requirements 

11.  Performance  requirements 

12.  Modes  of  operation 

13.  Technical  performance  measures 

14.  Physical  characteristics 

15.  Human  systems  integration 


Figure  4-3.  IEEE  PI 220  Requirements  Analysis  Task  Areas 
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Task  1.  Customer  Expectations 

Define  and  quantify  customer  expectations.  They 
may  come  from  any  of  the  eight  primary  func¬ 
tions,  operational  requirements  documents,  mis¬ 
sion  needs,  technology-based  opportunity,  direct 
communications  with  customer,  or  requirements 
from  a  higher  system  level.  The  purpose  of  this 
task  is  to  determine  what  the  customer  wants 
the  system  to  accomplish,  and  how  well  each 
function  must  be  accomplished.  This  should  in¬ 
clude  natural  and  induced  environments  in  which 
the  product(s)  of  the  system  must  operate  or  be 
used,  and  constraints  (e.g.  funding,  cost,  or  price 
objectives,  schedule,  technology,  non-develop- 
mental  and  reusable  items,  physical  characteris¬ 
tics,  hours  of  operation  per  day,  on-off  sequences, 
etc.). 

Task  2.  Project  and  Enterprise  Constraints 

Identify  and  define  constraints  impacting  design 
solutions.  Project  specific  constraints  can  include: 

•  Approved  specifications  and  baselines  devel¬ 
oped  from  prior  applications  of  the  Systems 
Engineering  Process, 

•  Costs, 

•  Updated  technical  and  project  plans, 

•  Team  assignments  and  structure, 

•  Control  mechanisms,  and 

•  Required  metrics  for  measuring  progress. 
Enterprise  constraints  can  include: 

•  Management  decisions  from  a  preceding 
technical  review, 

•  Enterprise  general  specifications, 

•  Standards  or  guidelines, 

•  Policies  and  procedures, 


•  Domain  technologies,  and 

•  Physical,  financial,  and  human  resource 
allocations  to  the  project. 

Task  3.  External  Constraints 

Identify  and  define  external  constraints  impacting 
design  solutions  or  implementation  of  the  Systems 
Engineering  Process  activities.  External  constraints 
can  include: 

•  Public  and  international  laws  and  regulations, 

•  Technology  base, 

•  Compliance  requirements:  Industry,  interna¬ 
tional,  and  other  general  specifications,  stan¬ 
dards,  and  guidelines  which  require  compliance 
for  legal,  interoperability,  or  other  reasons, 

•  Threat  system  capabilities,  and 

•  Capabilities  of  interfacing  systems. 

Task  4.  Operational  Scenarios 

Identify  and  define  operational  scenarios  that  scope 
the  anticipated  uses  of  system  product(s).  For  each 
operational  scenario,  define  expected: 

•  Interactions  with  the  environment  and  other 
systems,  and 

•  Physical  interconnectivities  with  interfacing 
systems,  platforms,  or  products. 

Task  5.  Measures  of  Effectiveness  and 
Suitability  (MOE/MOS) 

Identify  and  define  systems  effectiveness  measures 
that  reflect  overall  customer  expectations  and 
satisfaction.  MOEs  are  related  to  how  well  the 
system  must  perform  the  customer’s  mission.  Key 
MOEs  include  mission  performance,  safety,  op¬ 
erability,  reliability,  etc.  MOSs  are  related  to  how 
well  the  system  performs  in  its  intended  envi¬ 
ronment  and  includes  measures  of  supportabil- 
ity,  maintainability,  ease  of  use,  etc. 
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Task  6.  System  Boundaries 

Define  system  boundaries  including: 

•  Which  system  elements  are  under  design  con¬ 
trol  of  the  performing  activity  and  which  fall 
outside  of  their  control,  and 

•  The  expected  interactions  among  system  ele¬ 
ments  under  design  control  and  external  and / 
or  higher-level  and  interacting  systems  outside 
the  system  boundary  (including  open  systems 
approaches). 

Task  7.  Interfaces 

Define  the  functional  and  physical  interfaces  to 
external  or  higher-level  and  interacting  systems, 
platforms,  and/or  products  in  quantitative  terms 
(include  open  systems  approach).  Functional  and 
physical  interfaces  would  include  mechanical,  elec¬ 
trical,  thermal,  data,  control,  procedural,  and  other 
interactions.  Interfaces  may  also  be  considered 
from  an  internal/external  perspective.  Internal 
interfaces  are  those  that  address  elements  inside 
the  boundaries  established  for  the  system 
addressed.  These  interfaces  are  generally  identi¬ 
fied  and  controlled  by  the  contractor  responsible 
for  developing  the  system.  External  interfaces,  on 
the  other  hand,  are  those  which  involve  entity 
relationships  outside  the  established  boundaries, 
and  these  are  typically  defined  and  controlled  by 
the  government. 

Task  8.  Utilization  Environments 

Define  the  environments  for  each  operational 
scenario.  All  environmental  factors  (natural  or 
induced)  which  may  impact  system  performance 
must  be  identified  and  defined.  Environmental 
factors  include: 

•  Weather  conditions  (e.g.  rain,  snow,  sun,  wind, 
ice,  dust,  fog), 

•  Temperature  ranges, 

•  Topologies  (e.g.  ocean,  mountains,  deserts, 
plains,  vegetation), 


•  Biological  (e.g.  animal,  insects,  birds,  fungi), 

•  Time  (e.g.  day,  night,  dust),  and 

•  Induced  (e.g.  vibration,  electromagnetic,  chemi¬ 
cal). 

Task  9.  Life  Cycle  Process  Concepts 

Analyze  the  outputs  of  tasks  1-8  to  define  key  life 
cycle  process  requirements  necessary  to  develop, 
produce,  test,  distribute,  operate,  support,  train, 
and  dispose  of  system  products  under  develop¬ 
ment.  Use  integrated  teams  representing  the  eight 
primary  functions.  Focus  should  be  on  the  cost 
drivers  and  higher  risk  elements  that  are  antici¬ 
pated  to  impact  supportability  and  affordability 
over  the  useful  life  of  the  system. 

Task  10.  Functional  Requirements 

Define  what  the  system  must  accomplish  or  must 
be  able  to  do.  Functions  identified  through  require¬ 
ments  analysis  will  be  further  decomposed  during 
functional  analysis  and  allocation. 

Task  11.  Performance  Requirements 

Define  the  performance  requirements  for  each 
higher  level  function  performed  by  the  system. 
Primary  focus  should  be  placed  on  performance 
requirements  that  address  the  MOEs,  and  other 
key  performance  parameters  established  in  test 
plans  or  identified  as  interest  items  by  oversight 
authorities. 

Task  12.  Modes  of  Operation 

Define  the  various  modes  of  operation  for  the  sys¬ 
tem  products  under  development.  Conditions  (e.g. 
environmental,  configuration,  operational,  etc.) 
that  determine  the  modes  of  operation  should  be 
included  in  this  definition. 

Task  13.  Technical  Performance  Measures 
(TPMs) 

Identify  the  key  indicators  of  system  performance 
that  will  be  tracked  during  the  design  process. 
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Selection  of  TPMs  should  be  limited  to  critical 
technical  thresholds  and  goals  that,  if  not  met,  put 
the  project  at  cost,  schedule,  or  performance  risk. 
TPMs  involve  tracking  the  actual  versus  planned 
progress  of  key  performance  parameters  such  that 
the  manager  can  make  judgments  about  technical 
progress  on  a  by-exception  basis.  To  some  extent 
TPM  selection  is  phase  dependent.  They  must  be 
reconsidered  at  each  systems  engineering  process 
step  and  at  the  beginning  of  each  phase. 

Task  14.  Physical  Characteristics 

Identify  and  define  required  physical  characteris¬ 
tics  (e.g.  color,  texture,  size,  weight,  buoyancy) 
for  the  system  products  under  development.  Iden¬ 
tify  which  physical  characteristics  are  true  con¬ 
straints  and  which  can  be  changed,  based  on  trade 
studies. 

Task  15.  Human  Factors 

Identify  and  define  human  factor  considerations 
(e.g.  physical  space  limits,  climatic  limits,  eye 
movement,  reach,  ergonomics)  which  will  affect 
operation  of  the  system  products  under  develop¬ 
ment.  Identify  which  human  systems  integration 
are  constraints  and  which  can  be  changed  based 
on  trade  studies. 

Follow-on  Tasks 

The  follow-on  tasks  are  related  to  the  iterative 
nature  of  the  Systems  Engineering  Process: 


Integrate  Requirements: 

Take  an  integrated  team  approach  to  requirements 
determination  so  that  conflicts  among  and  between 
requirements  are  resolved  in  ways  that  result  in 
design  requirements  that  are  balanced  in  terms  of 
both  risk  and  affordability. 

Validate  Requirements: 

During  Functional  Analysis  and  Allocation,  vali¬ 
date  that  the  derived  functional  and  performance 
can  be  traced  to  the  operational  requirements. 

Verify  Requirements: 

•  Coordinate  design,  manufacturing,  deployment 
and  test  processes, 

•  Ensure  that  requirements  are  achievable  and 
testable, 

•  Verify  that  the  design-to-cost  goals  are 
achievable,  and 

•  Verify  that  the  functional  and  physical  archi¬ 
tectures  defined  during  Functional  Analysis/ 
Allocation  and  Synthesis  meet  the  integrated 
technical,  cost,  and  schedule  requirements 
within  acceptable  levels  of  risk. 
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5.1  INTRODUCTION 

The  purpose  of  this  systems  engineering  process 
activity  is  to  transform  the  functional,  performance, 
interface  and  other  requirements  that  were  identi¬ 
fied  through  requirements  analysis  into  a  coher¬ 
ent  description  of  system  functions  that  can  be  used 
to  guide  the  Design  Synthesis  activity  that  follows. 
The  designer  will  need  to  know  what  the  system 
must  do,  how  well,  and  what  constraints  will  limit 
design  flexibility. 

This  is  accomplished  by  arranging  functions  in 
logical  sequences,  decomposing  higher-level 
functions  into  lower-level  functions,  and  allocat¬ 
ing  performance  from  higher-  to  lower-level  func¬ 
tions.  The  tools  used  include  functional  flow  block 
diagrams  and  time  line  analysis;  and  the  product 
is  a  functional  architecture,  i.e.,  a  description  of 
the  system — but  in  terms  of  functions  and  perfor¬ 
mance  parameters,  rather  than  a  physical  description. 
Functional  Analysis  and  Allocation  facilitates 
traceability  from  requirements  to  the  solution  descrip¬ 
tions  that  are  the  outcome  of  Design  Synthesis. 

Functions  are  discrete  actions  (use  action  verbs) 
necessary  to  achieve  the  system’s  objectives.  These 
functions  may  be  stated  explicitly,  or  they  may  be 
derived  from  stated  requirements.  The  functions 
will  ultimately  be  performed  or  accomplished 
through  use  of  equipment,  personnel,  facilities, 
software,  or  a  combination. 

5.2  FUNCTIONAL  ANALYSIS  AND 
ALLOCATION 

Functional  and  performance  requirements  at  any 
level  in  the  system  are  developed  from  higher-level 


requirements.  Functional  Analysis  and  Allocation 
is  repeated  to  define  successively  lower-level  func¬ 
tional  and  performance  requirements,  thus  defin¬ 
ing  architectures  at  ever-increasing  levels  of  detail. 
System  requirements  are  allocated  and  defined  in 
sufficient  detail  to  provide  design  and  verification 
criteria  to  support  the  integrated  system  design. 

This  top-down  process  of  translating  system-level 
requirements  into  detailed  functional  and 
performance  design  criteria  includes: 

•  Defining  the  system  in  functional  terms,  then 
decomposing  the  top-level  functions  into 
subfunctions.  That  is,  identifying  at  succes¬ 
sively  lower  levels  what  actions  the  system  has 
to  do, 

•  Translating  higher-level  performance  require¬ 
ments  into  detailed  functional  and  performance 
design  criteria  or  constraints.  That  is,  iden¬ 
tifying  how  well  the  functions  have  to  be 
performed, 

•  Identifying  and  defining  all  internal  and  external 
functional  interfaces, 

•  Identifying  functional  groupings  to  minimize 
and  control  interfaces  (functional  partitioning), 

•  Determining  the  functional  characteristics  of 
existing  or  directed  components  in  the  system 
and  incorporating  them  in  the  analysis  and 
allocation, 

•  Examining  all  life  cycle  functions,  including 
the  eight  primary  functions,  as  appropriate  for 
the  specific  project, 
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•  Performing  trade  studies  to  determine  alterna¬ 
tive  functional  approaches  to  meet  requirements, 
and 

•  Revisiting  the  requirements  analysis  step  as 
necessary  to  resolve  functional  issues. 

The  objective  is  to  identify  the  functional,  per¬ 
formance,  and  interface  design  requirements;  it 
is  not  to  design  a  solution. ..yet! 

Functional  Partitioning 

Functional  partitioning  is  the  process  of  grouping 
functions  that  logically  fit  with  the  components 
likely  to  be  used,  and  to  minimize  functional  in¬ 
terfaces.  Partitioning  is  performed  as  part  of  func¬ 
tional  decomposition.  It  identifies  logical  group¬ 
ings  of  functions  that  enhance  the  use  of  modular 
components  and  open-system  designs.  Functional 
partitioning  is  also  useful  in  understanding  how 
existing  equipment  or  components  (including 
commercial)  will  function  with  or  within  the 
system. 


Requirements  Loop 

During  the  performance  of  the  Functional  Analysis 
and  Allocation  process,  it  is  expected  that  revisit¬ 
ing  the  requirements  analysis  process  will  be 
necessary.  This  is  caused  by  the  emergence  of 
functional  issues  that  will  require  re-examination 
of  the  higher-level  requirements.  Such  issues  might 
include  directed  components  or  standards  that 
cause  functional  conflict,  identification  of  a  revised 
approach  to  functional  sequencing,  or,  most  likely, 
a  conflict  caused  by  mutually  incompatible 
requirements. 

Figure  5-1  gives  an  overview  of  the  basic  param¬ 
eters  of  Functional  Analysis  and  Allocation.  The 
output  of  the  process  is  the  functional  architec¬ 
ture.  In  its  most  basic  form,  the  functional 
architecture  is  a  simple  hierarchical  decomposi¬ 
tion  of  the  functions  with  associated  performance 
requirements.  As  the  architecture  definition  is 
refined  and  made  more  specific  with  the  perfor¬ 
mance  of  the  activities  listed  in  Figure  5-1,  the 
functional  architecture  becomes  more  detailed  and 


Outputs: 

-  Functional  architecture  and  supporting  detail 

Inputs: 

-  Outputs  of  the  Requirements  Analysis 

Enablers: 

-  Multi-discipline  product  teams,  decision  database;  Tools  &  Models,  such  as  QFD,  Functional  Flow 
Block  Diagrams,  IDEF,  N2  charts,  Requirement  Allocation  Sheet,  Timelines,  Data  Flow  Diagrams, 
State/Mode  Diagrams,  Behavior  Diagrams 


Controls: 

-  Constraints;  GFE,  COTS,  &  Reusable  S/W;  System  concept 
&  subsystem  choices;  organizational  procedures 


Controls 

I 


Activities: 

-  Define  system  states  and  modes 

-  Define  system  functions  &  external  interfaces 

-  Define  functional  interfaces 

-  Allocate  performance  requirements  to  functions 

-  Analyze  performance 

-  Analyze  timing  and  resources 

-  Analyze  failure  mode  effects  and  criticality 

-  Define  fault  detection  and  recovery  behavior 

-  Integrate  functions 


Inputs 


Functional 
Analysis  & 
Allocation 


t 

Enablers 


Outputs 


Figure  5-1.  Functional  Analysis  and  Allocation 
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comprehensive.  These  activities  provide  a  func¬ 
tional  architecture  with  sufficient  detail  to  support 
the  Design  Synthesis.  They  are  performed  with 
the  aid  of  traditional  tools  that  structure  the  effort 
and  provide  documentation  for  traceability.  There 
are  many  tools  available.  The  following  are  tradi¬ 
tional  tools  that  represent  and  explain  the  primary 
tasks  of  Functional  Analysis  and  Allocation 
(several  of  these  are  defined  and  illustrated 
beginning  on  page  43): 

•  Functional  flow  block  diagrams  that  define  task 
sequences  and  relationships, 

•  IDEFO  diagrams  that  define  process  and  data 
flows, 

•  Timeline  analyses  that  define  the  time  sequence 
of  time  critical  functions,  and 

•  Requirements  allocation  sheets  that  identify 
allocated  performance  and  establish  traceability 
of  performance  requirements. 


Functional  Analysis  and  Allocation 


5.3  FUNCTIONAL  ARCHITECTURE 

The  functional  architecture  is  a  top-down  decom¬ 
position  of  system  functional  and  performance 
requirements.  The  architecture  will  show  not  only 
the  functions  that  have  to  be  performed,  but  also 
the  logical  sequencing  of  the  functions  and 
performance  requirements  associated  with  the 
functions.  It  also  includes  the  functional  descrip¬ 
tion  of  existing  and  government  furnished  items 
to  be  used  in  the  system.  This  may  require  reverse 
engineering  of  these  existing  components. 

The  functional  architecture  produced  by  the 
Functional  Analysis  and  Allocation  process  is  the 
detailed  package  of  documentation  developed  to 
analyze  the  functions  and  allocate  performance 
requirements.  It  includes  the  functional  flow  block 
diagrams,  timeline  sheets,  requirements  allocation 
sheets,  IDEFO  diagrams,  and  all  other  documen¬ 
tation  developed  to  describe  the  functional 
characteristics  of  the  system.  However,  there  is  a 
basic  logic  to  the  functional  architecture,  which  in 
its  preliminary  form  is  presented  in  the  example 
of  Figure  5-2.  The  Functional  Analysis  and 
Allocation  process  would  normally  begin  with 


Perform  Mission 


Transport  - 

■  50  km  90  min 


First  Level: 

Basic  Functional 
Requirement 


Second  Level: 
Transport  and 
communicate 
showing  as 
parallel  functions 


Third  Level: 
Showing  decom¬ 
position  of  the 
transport  func¬ 
tion 


A  Simple  Rule: 

Look  to  see  if  all  the  functions  are  verbs.  If  there  is  a  function  identified  as  a 
noun,  then  there  is  a  problem  with  the  understanding  of  the  functions. _ 
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Figure  5-2.  Functional  Architecture  Example 


39 


Systems  Engineering  Fundamentals 


Chapter  5 


the  IPT  drafting  such  a  basic  version  of  the  ar¬ 
chitecture.  This  would  generally  give  the  IPT  an 
understanding  of  the  scope  and  direction  of  the 
effort. 

Functional  Architecture  Example 

The  Marine  Corps  has  a  requirement  to  transport 
troops  in  squad-level  units  over  a  distance  of  50 
km.  Troops  must  be  transported  within  90  min¬ 
utes  from  the  time  of  arrival  of  the  transport  sys¬ 
tem.  Constant  communication  is  required  during 
the  transportation  of  troops.  Figure  5-2  illustrates 
a  preliminary  functional  architecture  for  this  simple 
requirement. 

5.4  SUMMARY  POINTS 

Functional  analysis  begins  with  the  output  of 
requirements  analysis  (that  is,  the  identification 
of  higher-level  functional  and  performance  require¬ 
ments).  Functional  Analysis  and  Allocation  con¬ 
sists  of  decomposition  of  higher-level  functions 
to  lower-levels  and  then  allocation  of  requirements 
to  those  functions. 


•  There  are  many  tools  available  to  support  the 
development  of  a  Functional  Architecture,  such 
as:  functional-flow  block  diagrams,  time-line 
analysis  sheet,  requirements  allocation  sheet, 
Integrated  Definition,  and  others. 

•  Use  of  the  tools  illustrated  in  this  chapter  is 
not  mandatory,  but  the  process  they  represent 
is: 

-  Define  task  sequences  and  relationships 
(FFBD), 

-  Define  process  and  data  flows  (IDEFO 
diagrams), 

-  Define  the  time  sequence  of  time-critical 
functions  (TLS),  and 

-  Allocate  performance  and  establish  trace- 
ability  of  performance  requirements  (RAS). 
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SUPPLEMENT  A 

FUNCTIONAL  FLOW 
BLOCK  DIAGRAM 


The  purpose  of  the  functional  flow  block  dia¬ 
gram  (FFBD)  is  to  describe  system  requirements 

in  functional  terms. 

Objectives 

The  FFBD  is  structured  to  ensure  that: 

•  All  life  cycle  functions  are  covered. 

•  All  elements  of  system  are  identified  and 
defined  (e.g.  prime  equipment,  training,  spare 
parts,  data,  software,  etc.). 

•  System  support  requirements  are  identified  to 
specific  system  functions. 


•  Proper  sequencing  of  activities  and  design 
relationships  are  established  including  critical 
design  interfaces. 

Characteristics 

The  FFBD  is  functionally  oriented — not  solution 
oriented.  The  process  of  defining  lower-level  func¬ 
tions  and  sequencing  relationships  is  often  referred 
to  as  functional  decomposition.  It  allows  traceabil¬ 
ity  vertically  through  the  levels.  It  is  a  key  step  in 
developing  the  functional  architecture  from  which 
designs  may  be  synthesized. 

Figure  5-3  shows  the  flow-down  structure  of  a  set 
of  FFBDs  and  Figure  5-4  shows  the  format  of  an 
FFBD. 


Figure  5-3.  FFBD  Traceability  and  Indenture 


41 


Systems  Engineering  Fundamentals 


Chapter  5 


Key  FFBD  Attributes 

Function  block:  Each  function  on  an  FFBD  should 
be  separate  and  be  represented  by  single  box  (solid 
line).  Each  function  needs  to  stand  for  definite, 
finite,  discrete  action  to  be  accomplished  by  system 
elements. 

Function  numbering:  Each  level  should  have  a 
consistent  number  scheme  and  provide  informa¬ 
tion  concerning  function  origin.  (E.g.  top  level — 
1.0,  2.0,  3.0,  etc;  first  indenture  (level  2) — 1.1, 
1.2,  1.3,  etc;  second  indenture  (level  3)— 1.1.1, 
1.1.2, 1.1.3,  etc.)  These  numbers  establish  identi¬ 
fication  and  relationships  that  will  carry  through 
all  Functional  Analysis  and  Allocation  activities 
and  facilitate  traceability  from  lower  to  top  levels. 

Functional  reference:  Each  diagram  should  con¬ 
tain  a  reference  to  other  functional  diagrams  by 
using  a  functional  reference  (box  in  brackets). 


Flow  connection:  Lines  connecting  functions 
should  only  indicate  function  flow  and  not  a  lapse 
in  time  or  intermediate  activity. 

Flow  direction:  Diagrams  should  be  laid  out  so 
that  the  flow  direction  is  generally  from  left  to 
right.  Arrows  are  often  used  to  indicate  functional 
flows. 

Summing  gates:  A  circle  is  used  to  denote  a  sum¬ 
ming  gate  and  is  used  when  AND/OR  is  present. 
AND  is  used  to  indicate  parallel  functions  and  all 
conditions  must  be  satisfied  to  proceed.  OR  is  used 
to  indicate  that  alternative  paths  can  be  satisfied 
to  proceed. 

GO  and  NO-GO  paths:  “G”  and  “bar  G”  are  used 
to  denote  “go”  and  “no-go”  conditions.  These  sym¬ 
bols  are  placed  adjacent  to  lines  leaving  a  particu¬ 
lar  function  to  indicate  alternative  paths. 
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SUPPLEMENT  B 

IDEFO 


Integration  Definition  for  Function  Modeling 
(IDEFO)  is  a  common  modeling  technique  for  the 
analysis,  development,  re-engineering,  and  inte¬ 
gration  of  information  systems;  business  processes; 
or  software  engineering  analysis.  Where  the  FFBD 
is  used  to  show  the  functional  flow  of  a  product, 
IDEFO  is  used  to  show  data  flow,  system  control, 
and  the  functional  flow  of  life  cycle  processes. 

IDEFO  is  capable  of  graphically  representing  a 
wide  variety  of  business,  manufacturing  and  other 
types  of  enterprise  operations  to  any  level  of  detail. 
It  provides  rigorous  and  precise  description,  and 
promotes  consistency  of  usage  and  interpretation. 
It  is  well-tested  and  proven  through  many  years  of 
use  by  government  and  private  industry.  It  can  be 
generated  by  a  variety  of  computer  graphics  tools. 
Numerous  commercial  products  specifically  sup¬ 
port  development  and  analysis  of  IDEFO  diagrams 
and  models. 

IDEFO  is  a  model  that  consists  of  a  hierarchical 
series  of  diagrams,  text,  and  glossary  cross-refer¬ 


enced  to  each  other.  The  two  primary  modeling 
components  are:  functions  (represented  on  a  dia¬ 
gram  by  boxes),  and  data  and  objects  that  inter¬ 
relate  those  functions  (represented  by  arrows). 
As  shown  by  Figure  5-5  the  position  at  which 
the  arrow  attaches  to  a  box  conveys  the  specific 
role  of  the  interface.  The  controls  enter  the  top 
of  the  box.  The  inputs,  the  data  or  objects  acted 
upon  by  the  operation,  enter  the  box  from  the 
left.  The  outputs  of  the  operation  leave  the 
right-hand  side  of  the  box.  Mechanism  arrows 
that  provide  supporting  means  for  performing 
the  function  join  (point  up  to)  the  bottom  of  the 
box. 

The  IDEF  process  starts  with  the  identification  of 
the  prime  function  to  be  decomposed.  This  func¬ 
tion  is  identified  on  a  “Top  Level  Context  Dia¬ 
gram,”  that  defines  the  scope  of  the  particular  IDEF 
analysis.  An  example  of  a  Top  Level  Context  Dia¬ 
gram  for  an  information  system  management  pro¬ 
cess  is  shown  in  Figure  5-6.  From  this  diagram 
lower-level  diagrams  are  generated.  An  example 
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Figure  5-5.  Integration  Definition  for  Function  Modeling  (IDEF)  Box  Format 
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of  a  derived  diagram,  called  a  “child”  in  IDEFO  supplement  IDEFO  for  data  intensive  systems.  The 
terminology,  for  a  life  cycle  function  is  shown  IDEFO  standard,  Federal  Information  Processing 
in  Figure  5-7.  Standards  Publication  183  (FIPS  183),  and  the 

IDEFlx  standard  (FPIS  184)  are  maintained  by 
An  associated  technique,  Integration  Definition  for  the  National  Institute  of  Standards  and  Technol- 
Information  Modeling  (IDEFlx),  is  used  to  ogy  (NIST). 
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Purpose:  The  assessment,  planning,  and  streamlining  of  information  management 
functions. 

Viewpoint:  The  information  Integration  Assessment  Team. 


Figure  5-6.  Top-Level  Context  Diagram 
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Figure  5-7.  IDEF  0  Diagram  Example 
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SUPPLEMENT  C 

TIMELINE  ANALYSIS 
SHEETS 


The  timeline  analysis  sheet  (TLS)  adds  detail  to 
defining  durations  of  various  functions.  It  defines 
concurrency,  overlapping,  and  sequential  relation¬ 
ships  of  functions  and  tasks.  It  identifies  time 
critical  functions  that  directly  affect  system  avail¬ 
ability,  operating  time,  and  maintenance  downtime. 
It  is  used  to  identify  specific  time-related  design 
requirements. 

The  TLS  includes  purpose  of  function  and  the 
detailed  performance  characteristics,  criticality 


of  function,  and  design  constraints.  It  identifies 
both  quantitative  and  qualitative  performance  re¬ 
quirements.  Initial  resource  requirements  are 
identified. 

Figure  5-6  shows  an  example  of  a  TLS.  The  time 
required  to  perform  function  3.1  and  its 
subfunctions  are  presented  on  a  bar  chart  showing 
how  the  timelines  relate.  (Function  numbers  match 
the  FFBD.) 


Function  3.1  Establish  and  maintain  vehicle 
readiness  from  35  hrs  to  2  hrs  prior  to  launch 


Function 


Provide  ground  power 


Provide  vehicle  air  conditioning 


Install  and  connect  batteries 


Install  ordnance 


Perform  stray  voltage  checks  and 
connect  ordnance 


Load  fuel  tanks 


Load  oxidizer  tanks 


Activate  guidance  system 


Establish  propulsion  flight  pressure 


Telemetry  system  “on” 


Figure  5-8.  Time  Analysis  Sheet 
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SUPPLEMENT  D 

REQUIREMENTS 
ALLOCATION  SHEET 


The  Requirements  Allocation  Sheet  documents 
the  connection  between  allocated  functions,  al¬ 
located  performance  and  the  physical  system.  It 
provides  traceability  between  Functional  Analy¬ 
sis  and  Allocation  and  Design  Synthesis,  and 


shows  any  disconnects.  It  is  a  major  tool  in  main¬ 
taining  consistency  between  functional  architec¬ 
tures  and  designs  that  are  based  on  them.  (Func¬ 
tion  numbers  match  the  FFBD.) 


Requirements 
Allocation  Sheet 

Functional  Flow  Diagram  Title  and  No.  2.58.4 
Provide  Guidance  Compartment  Cooling 

Equipment 

Identificatio 

n 

Function  Name 
and  No. 

Functional  Performance  and 

Design  Requirements 

Facility 

Rqmnts 

Nomen¬ 

clature 

Cl  or  Detail 
Spec  No. 

2.58.4  Provide 
Guidance 
Compartment 
Cooling 

The  temperature  in  the  guidance 
compartment  must  be  maintained  at  the 
initial  calibration  temperature  of  +0.2  Degrees 
F.  The  initial  calibration  temperature  of  the 
compartment  will  be  between  66.5  and  68.5 
Degrees  F. 

2.58.4.1  Provide 
liquid 

Chilled  Coolant 
(Primary) 

A  storage  capacity  for  65  gallons  of  chilled 
coolant  (deionized  water)  is  required.  The 
temperature  of  the  stored  coolant  must  be 
monitored  continuously.  The  stored  coolant 
must  be  maintained  within  a  temperature 
range  of  40-50  Degrees  F.  for  an  indefinite 
period  of  time.  The  coolant  supplied  must 
be  free  of  obstructive  particles  0.5  micron  at 
all  times. 

k 

Figure  5-9.  Requirements  Allocation  Sheet  (Example) 
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DESIGN  SYNTHESIS 


6.1  DESIGN  DEVELOPMENT  tive  of  design  synthesis  is  to  combine  and  re¬ 

structure  hardware  and  software  components  in 
Design  Synthesis  is  the  process  by  which  con-  such  a  way  as  to  achieve  a  design  solution  cap- 

cepts  or  designs  are  developed  based  on  the  func-  able  of  satisfying  the  stated  requirements.  During 

tional  descriptions  that  are  the  products  of  Func-  concept  development,  synthesis  produces  sys- 

tional  Analysis  and  Allocation.  Design  synthe-  tern  concepts  and  establishes  basic  relation- 

sis  is  a  creative  activity  that  develops  a  physical  ships  among  the  subsystems.  During  preliminary 

architecture  (a  set  of  product,  system,  and/or  and  detailed  design,  subsystem  and  component 

software  elements)  capable  of  performing  the  descriptions  are  elaborated,  and  detailed  inter- 

required  functions  within  the  limits  of  the  per-  faces  between  all  system  components  are  de- 

formance  parameters  prescribed.  Since  there  may  fined, 

be  several  hardware  and/or  software  architec¬ 
tures  developed  to  satisfy  a  given  set  of  func-  The  physical  architecture  forms  the  basis  for 
tional  and  performance  requirements,  synthesis  design  definition  documentation,  such  as,  speci- 
sets  the  stage  for  trade  studies  to  select  the  best  fications,  baselines,  and  work  breakdown  struc- 
among  the  candidate  architectures.  The  objec-  tures.  Figure  6-1  gives  an  overview  of  the  basic 

parameters  of  the  synthesis  process. 


•  Outputs: 

“  Physical  Architecture  (Product  Elements  and  Software  Code) 

-  Decision  Database 

•  Inputs: 

-  Functional  Architecture 

•  Enablers: 

-  IPTs,  Decision  Database,  Automated  Tools,  Models 

•  Controls: 

-  Constraints;  GFE,  COTS,  &  Reusable  S/W;  System  concept 

Controls 

1 

&  subsystem  choices;  organizational  procedures 

▼ 

«  Activities: 

-  Allocate  functions  and  constraints  to  system  elements 

|  ■  V;'  - 

-  Synthesize  system  element  alternatives  Inputs 

■■^^Outputs 

-  Assess  technology  alternatives  w 

I  Synttiesls 

-  Define  physical  interfaces 

t  . .  .  u  •  ■ '  v  .. 

-  Define  system  product  WBS 

-  Develop  life  cycle  techniques  and  procedures 

-  Integrate  system  elements 

t 

-  Select  preferred  concept/design 

Enablers 

Figure  6-1.  Design  Synthesis 
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Characteristics 

Physical  architecture  is  a  traditional  term.  De¬ 
spite  the  name,  it  includes  software  elements  as 
well  as  hardware  elements.  Among  the  charac¬ 
teristics  of  the  physical  architecture  (the  primary 
output  of  Design  Synthesis)  are  the  following: 

•  The  correlation  with  functional  analysis 
requires  that  each  physical  or  software  com¬ 
ponent  meets  at  least  one  (or  part  of  one)  func¬ 
tional  requirement,  though  any  component  can 
meet  more  than  one  requirement, 

•  The  architecture  is  justified  by  trade  studies 
and  effectiveness  analyses, 

•  A  product  Work  Breakdown  Structure  (WBS) 
is  developed  from  the  physical  architecture, 

•  Metrics  are  developed  to  track  progress 
among  key  performance  parameters,  and 

•  All  supporting  information  is  documented  in 
a  database. 

Modular  Designs 

Modular  designs  are  formed  by  grouping  com¬ 
ponents  that  perform  a  single  independent  func¬ 
tion  or  single  logical  task;  have  single  entry  and 
exit  points;  and  are  separately  testable.  Group¬ 
ing  related  functions  facilitates  the  search  for 
modular  design  solutions  and  furthermore  in¬ 
creases  the  possibility  that  open-systems  ap¬ 
proaches  can  be  used  in  the  product  architec¬ 
ture. 

Desirable  attributes  of  the  modular  units  include 
low  coupling,  high  cohesion,  and  low 
connectivity.  Coupling  between  modules  is  a 
measure  of  their  interdependence,  or  the  amount 
of  information  shared  between  two  modules. 
Decoupling  modules  eases  development  risks  and 
makes  later  modifications  easier  to  implement. 
Cohesion  (also  called  binding)  is  the  similarity 
of  tasks  performed  within  the  module.  High 
cohesion  is  desirable  because  it  allows  for  use 


of  identical  or  like  (family  or  series)  components, 
or  for  use  of  a  single  component  to  perform  mul¬ 
tiple  functions.  Connectivity  refers  to  the  rela¬ 
tionship  of  internal  elements  within  one  module 
to  internal  elements  within  another  module.  High 
connectivity  is  undesirable  in  that  it  creates  com¬ 
plex  interfaces  that  may  impede  design,  devel¬ 
opment,  and  testing. 

Design  Loop 

The  design  loop  involves  revisiting  the  functional 
architecture  to  verify  that  the  physical  architec¬ 
ture  developed  is  consistent  with  the  functional 
and  performance  requirements.  It  is  a  mapping 
between  the  functional  and  physical  architectures. 
Figure  6-2  shows  an  example  of  a  simple  physi¬ 
cal  architecture  and  how  it  relates  to  the  func¬ 
tional  architecture.  During  design  synthesis,  re- 
evaluation  of  the  functional  analysis  may  be 
caused  by  the  discovery  of  design  issues  that 
require  re-examination  of  the  initial  decomposi¬ 
tion,  performance  allocation,  or  even  the  higher- 
level  requirements.  These  issues  might  include 
identification  of  a  promising  physical  solution 
or  open-system  opportunities  that  have  different 
functional  characteristics  than  those  foreseen  by 
the  initial  functional  architecture  requirements. 

6.2  SYNTHESIS  TOOLS 

During  synthesis,  various  analytical,  engineer¬ 
ing,  and  modeling  tools  are  used  to  support  and 
document  the  design  effort.  Analytical  devices 
such  as  trade  studies  support  decisions  to  opti¬ 
mize  physical  solutions.  Requirements  Alloca¬ 
tion  Sheets  (RAS)  provide  traceability  to  the 
functional  and  performance  requirements.  Simple 
descriptions  like  the  Concept  Design  Sheet 
(CDS)  help  visualize  and  communicate  the  sys¬ 
tem  concept.  Logic  models,  such  as  the  Sche¬ 
matic  Block  Diagram  (SBD),  establish  the  de¬ 
sign  and  the  interrelationships  within  the  sys¬ 
tem. 

Automated  engineering  management  tools  such 
as  Computer-Aided  Design  (CAD),  Computer- 
Aided-Systems  Engineering  (CASE),  and  the 
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Figure  6-2.  Functional/Physical  Matrix 


Computer-Aided-Engineering  (CAE)  can  help 
organize,  coordinate  and  document  the  design 
effort.  Computer-Aided  Design  (CAD)  gener¬ 
ates  detailed  documentation  describing  the  prod¬ 
uct  design  including  SBDs,  detailed  drawings, 
three-dimensional  and  solid  drawings,  and  it 
tracks  some  technical  performance  measure¬ 
ments.  CAD  can  provide  significant  input  for 
virtual  modeling  and  simulations.  It  also  pro¬ 
vides  a  common  design  database  for  integrated 
design  developments.  Computer-Aided  Engineer¬ 
ing  can  provide  system  requirements  and  per¬ 
formance  analysis  in  support  of  trade  studies, 
analysis  related  to  the  eight  primary  functions, 
and  cost  analyses.  Computer-Aided  Systems  En¬ 
gineering  can  provide  automation  of  technical 
management  analyses  and  documentation. 


Modeling 

Modeling  techniques  allow  the  physical  product 
to  be  visualized  and  evaluated  prior  to  design 
decisions.  Models  allow  optimization  of  hard¬ 
ware  and  software  parameters,  permit  perfor¬ 
mance  predictions  to  be  made,  allow  operational 
sequences  to  be  derived,  and  permit  optimum 
allocation  of  functional  and  performance  require¬ 
ments  among  the  system  elements.  The  tradi¬ 
tional  logical  prototyping  used  in  Design  Syn¬ 
thesis  is  the  Schematic  Block  Diagram. 
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6.3  SUMMARY  POINTS 

•  Synthesis  begins  with  the  output  of  Functional 
Analysis  and  Allocation  (the  functional  ar¬ 
chitecture).  The  functional  architecture  is 
transformed  into  a  physical  architecture  by 
defining  physical  components  needed  to  per¬ 
form  the  functions  identified  in  Functional 
Analysis  and  Allocation. 


•  Many  tools  are  available  to  support  the 
development  of  a  physical  architecture: 

-  Define  and  depict  the  system  concept 
(CDS), 

-  Define  and  depict  components  and  their 
relationships  (SBD),  and 

-  Establish  traceability  of  performance 
requirements  to  components  (RAS). 

•  Specifications  and  the  product  WBS  are  de¬ 
rived  from  the  physical  architecture. 
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SUPPLEMENT  A 

CONCEPT  DESCRIPTION 
SHEET 

The  Concept  Description  Sheet  describes  (in  tex-  tem  will  be  integrated  to  meet  the  performance 
tual  or  graphical  form)  the  technical  approach  and  functional  requirementst.  It  is  generally  used 
or  the  design  concept,  and  shows  how  the  sys-  in  early  concept  design  to  show  system  concepts. 


Figure  6-3.  Concept  Description  Sheet  Example 


53 


Systems  Engineering  Fundamentals 


Chapter  6 


SUPPLEMENT  B 


SCHEMATIC  BLOCK  DIAGRAMS 


The  Schematic  Block  Diagram  (SBD)  depicts 
hardware  and  software  components  and  their 
interrelationships.  They  are  developed  at  suc¬ 
cessively  lower  levels  as  analysis  proceeds  to 
define  lower-level  functions  within  higher-level 
requirements.  These  requirements  are  further 
subdivided  and  allocated  using  the  Requirements 
Allocation  Sheet  (RAS).  SBDs  provide  visibil¬ 
ity  of  related  system  elements,  and  traceability 
to  the  RAS,  FFBD,  and  other  system  engineer¬ 
ing  documentation.  They  describe  a  solution  to 
the  functional  and  performance  requirements 
established  by  the  functional  architecture;  show 
interfaces  between  the  system  components  and 
between  the  system  components  and  other  sys¬ 


tems  or  subsystems;  support  traceability  between 
components  and  their  functional  origin;  and  pro¬ 
vide  a  valuable  tool  to  enhance  configuration 
control.  The  SBD  is  also  used  to  develop  Inter¬ 
face  Control  Documents  (ICDs)  and  provides 
an  overall  understanding  of  system  operations. 

A  simplified  SBD,  Figure  6-4,  shows  how  com¬ 
ponents  and  the  connection  between  them  are 
presented  on  the  diagram.  An  expanded  version 
is  usually  developed  which  displays  the  detailed 
functions  performed  within  each  component  and 
a  detailed  depiction  of  their  interrelationships. 
Expanded  SBDs  will  also  identify  the  WBS 
numbers  associated  with  the  components. 
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SUPPLEMENT  C 

REQUIREMENTS  ALLOCATION 

SHEET 


The  Requirements  Allocation  Sheet  initiated  in 
Functional  Analysis  and  Allocation  is  expanded 
in  Design  Synthesis  to  document  the  connec¬ 
tion  between  functional  requirements  and  the 
physical  system.  It  provides  traceability  between 


the  Functional  Analysis  and  Allocation  and  Syn¬ 
thesis  activities.  It  is  a  major  tool  in  maintaining 
consistency  between  functional  architectures  and 
the  designs  that  are  based  on  them.  (Cl  numbers 
match  the  WBS.) 


Requirements 
Allocation  Sheet 

Functional  Flow  Diagram  Title  and  No.  2.58.4 
Provide  Guidance  Compartment  Cooling 

Equipment 

Identification 

Function  Name 
and  No. 

Functional  Performance  and 

Design  Requirements 

Facility 

Rqmnts 

Nomenclature 

Cl  or  Detail 
Spec  No. 

2.58.4  Provide 
Guidance 
Compartment 
Cooling 

The  temperature  in  the  guidance 
compartment  must  be  maintained 
at  the  initial  calibration  tempera¬ 
ture  of  +0.2  Degrees  F.The  initial 
calibration  temperature  of  the 
compartment  will  be  between  66.5 
and  68.5  Degrees  F. 

Guidance  Compart¬ 
ment  Cooling 
System 

3.54.5 

2.58.4.1  Provide 
Chilled  Coolant 
(Primary) 

j 

A  storage  capacity  for  65  gallons  of 
chilled  liquid  coolant  (deionized 
water)  is  required.  The  temperature 
of  the  stored  coolant  must  be 
monitored  continuously.  The  stored 
coolant  must  be  maintained  within 
a  temperature  range  of  40-50 
Degrees  F.  for  an  indefinite  period 
of  time.  The  coolant  supplied  must 
be  free  of  obstructive  particles  0.5 
micron  at  all  times. 

Guidance  Compart¬ 
ment  Coolant 
Storage  Subsystem 

3.54.5.1 

Figure  6-5.  Requirements  Allocation  Sheet  (Example) 
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7.1  GENERAL 

The  Verification  process  confirms  that  Design 
Synthesis  has  resulted  in  a  physical  architecture 
that  satisfies  the  system  requirements.  Verification 
represents  the  intersection  of  systems  engineering 
and  test  and  evaluation. 

Verification  Objectives 

The  objectives  of  the  Verification  process  include 
using  established  criteria  to  conduct  verification 
of  the  physical  architecture  (including  software  and 
interfaces)  from  the  lowest  level  up  to  the  total  sys¬ 
tem  to  ensure  that  cost,  schedule,  and  performance 


requirements  are  satisfied  with  acceptable  levels 
of  risk.  Further  objectives  include  generating  data 
(to  confirm  that  system,  subsystem,  and  lower  level 
items  meet  their  specification  requirements)  and 
validating  technologies  that  will  be  used  in  system 
design  solutions. 

Verification  Activities 

System  design  solutions  are  verified  by  analysis, 
examination,  demonstration,  or  test.  Required 
defining  characteristics,  such  as  key  performance 
parameters  (KPPs)  are  verified  by  demonstration 
and  test.  Where  total  verification  by  test  is  not  fea¬ 
sible,  testing  is  used  to  verify  key  characteristics 


Figure  7-1.  Design  Synthesis 
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and  assumptions  used  in  design  analysis  or  simu¬ 
lation.  Validated  models  and  simulation  tools  are 
included  as  analytical  verification  methods  that 
complement  other  methods.  The  focus  and  nature 
of  verification  activities  change  as  designs  progress 
from  concept  to  detailed  designs  to  physical 
products. 

During  earlier  design  stages,  verification  focuses 
on  proof  of  concept  for  system,  subsystem  and 
component  levels.  During  later  stages,  as  the  prod¬ 
uct  definition  effort  proceeds,  the  focus  turns  to 
verifying  that  the  system  meets  the  customer 
requirements.  As  shown  by  Figure  7-1,  design  is  a 
top-down  process  while  the  verification  activity  is 
a  bottom-up  process.  Components  will  be  fabri¬ 
cated  and  tested  prior  to  the  subsystems.  Sub¬ 
systems  will  be  fabricated  and  tested  prior  to  the 
completed  system. 

Performance  Verification 

Performance  requirements  must  be  objectively 
verifiable,  i.e.,  the  requirement  must  be  measur¬ 
able.  Where  appropriate,  Technical  Performance 
Measurements  (TPM)  and  other  management 
metrics  are  used  to  provide  insight  on  progress 
toward  meeting  performance  goals  and  require¬ 
ments.  IEEE  Standard  P1220  provides  a  structure 
for  Verification  activity.  As  shown  in  Figure  7-2 
the  structure  is  comprehensive  and  provides  a  good 
starting  point  for  Verification  planning. 

7.2  DOD  TEST  &  EVALUATION 

DoD  Test  &  Evaluation  (T&E)  policies  and  pro¬ 
cedures  directly  support  the  system  engineering 
process  of  Verification.  Testing  is  the  means  by 
which  objective  judgments  are  made  regarding 
the  extent  to  which  the  system  meets,  exceeds, 
or  fails  to  meet  stated  objectives.  The  purpose 
of  evaluation  is  to  review,  analyze,  and  assess 
data  obtained  from  testing  and  other  means  to 
aid  in  making  systematic  decisions.  The  purpose 
of  DoD  Test  &  Evaluation  is  to  verify  technical 
performance,  operational  effectiveness,  opera¬ 
tional  suitability;  and  it  provides  essential 
information  in  support  of  decision  making. 


Common  Types  of  T&E  in  DoD 

T&E  policy  requires  developmental  tests.  They 
confirm  that  technical  requirements  have  been 
satisfied,  and  independent  analysis  and  tests  verify 
the  system’s  operational  effectiveness  and  suita¬ 
bility.  DoD  T&E  traditionally  and  by  directive  is 
categorized  as: 

•  Developmental  T&E  which  focuses  primarily 
on  technical  achievement, 

•  Operational  T&E  which  focuses  on  operational 
effectiveness  and  suitability  and  includes  Early 
Operational  Assessments  (EOA),  Operational 
Assessment  (OA),  Initial  Operational  Test  & 
Evaluation  (IOT&E),  and  Follow-On 
Operational  Test  &  Evaluation  (FOT&E),  and 

•  Live  Fire  T&E  which  provides  assessment  of 
the  vulnerability  and  lethality  of  a  system  by 
subjecting  it  to  real  conditions  comparable  to 
the  required  mission. 

Test  &  Evaluation  Management 

The  program  office  plans  and  manages  the  test 
effort  to  ensure  testing  is  timely,  efficient,  com¬ 
prehensive  and  complete — and  that  test  results  are 
converted  into  system  improvements.  Test  plan¬ 
ning  will  determine  the  effectiveness  of  the 
verification  process.  Like  all  systems  engineering 
planning  activities,  careful  attention  to  test 
planning  can  reduce  program  risk.  The  key  test 
planning  document  is  the  Test  and  Evaluation 
Master  Plan.  This  document  lays  out  the  objec¬ 
tives,  schedule,  and  resources  reflecting  program 
office  and  operational  test  organization  planning 
decisions.  To  ensure  integration  of  this  effort,  the 
program  office  organizes  a  Test  Planning  Work 
Group  (TPWG)  or  Test  Working  Level  IPT  (WIPT) 
to  coordinate  the  test  planning  effort. 

Test  Planning  Work  Group  /Test  WIPT 

The  TPWG  /  Test  WIPT  is  intended  to  facilitate 
the  integration  of  test  requirements  and  activities 
through  close  coordination  between  the  members 
who  represent  the  material  developer,  designer 
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Figure  7-2.  Verification  Tasks 


community,  logistic  community,  user,  operational 
tester,  and  other  stakeholders  in  the  system  de¬ 
velopment.  The  team  outlines  test  needs  based 
on  system  requirements,  directs  test  design,  de¬ 
termines  needed  analyses  for  each  test,  identi¬ 
fies  potential  users  of  test  results,  and  provides 
rapid  dissemination  of  test  and  evaluation  re¬ 
sults. 

Test  &  Evaluation  Master  Plan  (TEMP) 

The  Test  and  Evaluation  Master  Plan  is  a  manda¬ 
tory  document  prepared  by  the  program  office.  The 
operational  test  organization  reviews  it  and 


provides  the  operational  test  planning  for  inclu¬ 
sion.  The  TEMP  is  then  negotiated  between  the 
program  office  and  operational  test  organization. 
After  differences  are  resolved,  it  is  approved  at 
appropriate  high  levels  in  the  stakeholder  organi¬ 
zations.  After  approval  it  becomes  binding  on  man¬ 
agers  and  designers  (similar  to  the  binding  nature 
of  the  Operational  Requirements  Document 
(ORD)). 

The  TEMP  is  a  valuable  Verification  tool  that 
provides  an  excellent  template  for  technology, 
system,  and  major  subsystem-level  Verification 
planning.  The  TEMP  includes  a  reaffirmation 
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of  the  user  requirements,  and  to  an  extent,  an 
interpretation  of  what  those  requirements  mean 
in  various  operational  scenarios.  Part  I  of  the 
required  TEMP  format  is  System  Introduction, 
which  provides  the  mission  description,  threat 
assessment,  Measures  of  Effectiveness  and  Suit¬ 
ability,  a  system  description,  and  an  identifica¬ 
tion  of  critical  technical  parameters.  Part  II,  In¬ 
tegrated  Test  Program  Summary,  provides  an 
integrated  test  program  schedule  and  a  descrip¬ 
tion  of  the  overall  test  management  process.  Part 
III,  Developmental  Test  &  Evaluation  Outline, 
lays  out  an  overview  of  DT&E  efforts  and  a 
description  of  future  DT&E.  Part  IV,  Operational 
Test  &  Evaluation  Outline,  is  provided  by  the 
operational  test  organization  and  includes  an 
OT&E  overview,  critical  operational  issues,  fu¬ 
ture  OT&E  description,  and  LFT&E  description. 
Part  V,  Test  &  Evaluation  Resource  Summary, 
identifies  the  necessary  physical  resources  and 
activity  responsibilities.  This  last  part  includes 
such  items  as  test  articles,  test  sites,  test  instru¬ 
mentation,  test  support  equipment,  threat 
representation,  test  targets  and  other  expendables, 
operational  force  test  support,  simulations,  models, 
test-beds,  special  requirements,  funding,  and  training. 

Key  Performance  Parameters 

Every  system  will  have  a  set  of  key  performance 
parameters  (KPPs)  that  are  the  performance  char¬ 
acteristics  that  must  be  achieved  by  the  design  so¬ 
lution.  They  flow  from  the  operational  require¬ 
ments  and  the  resulting  derived  measures  of  ef¬ 
fectiveness  (MOEs).  They  can  be  identified  by  the 
user,  the  decision  authority,  or  the  operational 
tester.  They  are  documented  in  the  Test  and  Evalu¬ 
ation  Master  Plan. 

Developmental  Test  &  Evaluation 

The  Developmental  Test  &  Evaluation  (DT&E) 
verifies  that  the  design  solution  meets  the  system 
technical  requirements  and  the  system  is  prepared 
for  successful  OT&E.  DT&E  activities  assess 
progress  toward  resolving  critical  operational 
issues,  the  validity  of  cost-performance  tradeoff 
decisions,  the  mitigation  of  acquisition  technical 
risk,  and  the  achievement  of  system  maturity. 


Developmental  Test  and  Evaluation  efforts: 

•  Identify  potential  operational  and  technologi¬ 
cal  capabilities  and  limitations  of  the  alterna¬ 
tive  concepts  and  design  options  being  pursued; 

•  Support  the  identification  of  cost-performance 
tradeoffs  by  providing  analyses  of  the 
capabilities  and  limitations  of  alternatives; 

•  Support  the  identification  and  description  of 
design  technical  risks; 

•  Assess  progress  toward  resolving  Critical 
Operational  Issues,  mitigating  acquisition 
technical  risk,  achieving  manufacturing  process 
requirements  and  system  maturity; 

•  Assess  validity  of  assumptions  and  analysis 
conclusions;  and 

•  Provide  data  and  analysis  to  certify  the  system 
ready  for  operational  test  and  evaluation,  live- 
fire  testing  and  other  required  certifications. 

Figure  7-3  highlights  some  of  the  more  signifi¬ 
cant  DT&E  focus  areas  and  where  they  fit  in  the 
acquisition  life  cycle. 

Live  Fire  Test  &  Evaluation 

Live  Fire  Test  &  Evaluation  (LFT&E)  is  performed 
on  any  ACAT  I  or  II  level  weapon  system  that 
includes  features  designed  to  provide  protection 
to  the  system  or  its  users  in  combat.  It  is  conducted 
on  a  production  configured  article  to  provide 
information  concerning  potential  user  casualties, 
vulnerabilities,  and  lethality.  It  provides  data  that 
can  establish  the  system’s  susceptibility  to  attack 
and  performance  under  realistic  combat  conditions. 

Operational  Test  &  Evaluation 

Operational  Test  &  Evaluation  (OT&E)  programs 
are  structured  to  determine  the  operational  effec¬ 
tiveness  and  suitability  of  a  system  under  realistic 
conditions,  and  to  determine  if  the  minimum 
acceptable  operational  performance  requirements 
as  specified  in  the  ORD  and  reflected  by  the  KPPs 
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Figure  7-3.  DT&E  During  System  Acquisition 


have  been  satisfied.  Operational  Test  &  Evalua¬ 
tion  uses  threat-representative  forces  whenever 
possible,  and  employs  typical  users  to  operate  and 
maintain  the  system  or  item  under  conditions  simu¬ 
lating  both  combat  stress  and  peacetime  conditions. 
Operational  tests  will  use  production  or  produc¬ 
tion-representative  articles  for  the  operational  tests 
that  support  the  full-rate  production  decision.  Live 
Fire  Tests  are  usually  performed  during  the  opera¬ 
tional  testing  period.  Figure  7-4  shows  the  major 
activities  associated  with  operational  testing  and 
where  they  fit  in  the  DoD  acquisition  life  cycle. 

OT&E  Differences 

Though  the  overall  objective  of  both  DT&E  and 
OT&E  is  to  verify  the  effectiveness  and  suitabil¬ 
ity  of  the  system,  there  are  distinct  differences  in 
their  specific  objects  and  focus.  DT&E  primarily 
focuses  on  verifying  system  technical  require¬ 
ments,  while  OT&E  focuses  on  verifying  opera¬ 


tional  requirements.  DT&E  is  a  program  office 
responsibility  that  is  used  to  develop  the  design. 
OT&E  is  an  independent  evaluation  of  design 
maturity  that  is  used  to  determine  if  the  program 
should  proceed  to  full  rate  production.  Figure  7-5 
lists  the  major  differences  between  the  two. 

7.3  SUMMARY  POINTS 

The  Verification  activities  of  the  Systems  Engi¬ 
neering  Process  are  performed  to  verify  that  physi¬ 
cal  design  meets  the  system  requirements. 

•  DoD  Test  and  Evaluation  policy  supports  the 
verification  process  through  a  sequence  of 
developmental,  operational,  and  live-fire  tests, 
analyses,  and  assessments.  The  primary  man¬ 
agement  tools  for  planning  and  implementing 
the  T&E  effort  are  the  TEMP  and  the  integrated 
planning  team. 
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Figure  7-4.  OT&E  During  System  Acquisition 


Development  Tests 

•  Controlled  by  program  manager 

•  One-on-one  tests 

•  Controlled  environment 

•  Contractor  environment 

•  Trained,  experienced  operators 

•  Precise  performance  objectives  and 
threshold  measurements 

•  Test  to  specification 

•  Developmental,  engineering,  or  production 
representative  test  article 


Operational  Tests 

•  Controlled  by  independent  agency 

•  Many-on-many  tests 

•  Realistic/tactical  environment  with 
operational  scenario 

•  No  system  contractor  involvement 

•  User  troops  recently  trained 

•  Performance  measures  of  operational 
effectiveness  and  suitability 

•  Test  to  operational  requirements 

•  Production  representative  test  article 


Figure  7-5.  DT/OT  Comparison 
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SYSTEMS  ENGINEERING 
PROCESS  OUTPUTS 


1.1  DOCUMENTING  REQUIREMENTS 
AND  DESIGNS 

Outputs  of  the  systems  engineering  process  con¬ 
sist  of  the  documents  that  define  the  system 
requirements  and  design  solution.  The  physical 
architecture  developed  through  the  synthesis 
process  is  expanded  to  include  enabling  products 
and  services  to  complete  the  system  architecture. 
This  system  level  architecture  then  becomes  the 
reference  model  for  further  development  of  sys¬ 
tem  requirements  and  documents.  System  engi¬ 
neering  process  outputs  include  the  system  and 
configuration  item  architectures,  specifications, 
and  baselines,  and  the  decision  database. 

Outputs  are  dependent  on  the  level  of  development. 
They  become  increasingly  technically  detailed  as 
system  definition  proceeds  from  concept  to 
detailed  design.  As  each  stage  of  system  defini¬ 
tion  is  achieved,  the  information  developed  forms 
the  input  for  succeeding  applications  of  the  system 
engineering  process. 

Architectures:  System/Configuration  Item 

The  System  Architecture  describes  the  entire  sys¬ 
tem.  It  includes  the  physical  architecture  produced 
through  design  synthesis  and  adds  the  enabling 
products  and  services  required  for  life  cycle 
employment,  support,  and  management.  MIL- 
HDBK-881,  Work  Breakdown  Structures,  provides 
reference  models  for  weapon  systems  architectures. 
As  shown  by  Figure  8-1,  Military  Handbook 
(MIL-HDBK)-881  illustrates  the  first  three  lev¬ 
els  of  typical  system  architectures.  Program  Of¬ 
fices  can  use  MIL-HDBK-881  templates  during 
system  definition  to  help  develop  a  top-level  ar¬ 
chitecture  tailored  to  the  needs  of  the  specific 


system  considered.  The  design  contractor  will 
normally  develop  the  levels  below  these  first 
three.  Chapter  9  of  this  text  describes  the  Work 
Breakdown  Structure  in  more  detail. 

Specifications 

A  specification  is  a  document  that  clearly  and 
accurately  describes  the  essential  technical  require¬ 
ments  for  items,  materials,  or  services  including 
the  procedures  by  which  it  can  be  determined  that 
the  requirements  have  been  met.  Specifications 
help  avoid  duplication  and  inconsistencies,  allow 
for  accurate  estimates  of  necessary  work  and 
resources,  act  as  a  negotiation  and  reference  docu¬ 
ment  for  engineering  changes,  provide  documen¬ 
tation  of  configuration,  and  allow  for  consistent 
communication  among  those  responsible  for  the 
eight  primary  functions  of  Systems  Engineering. 
They  provide  integrated  product  teams  a  precise 
idea  of  the  problem  to  be  solved  so  that  they  can 
efficiently  design  the  system  and  estimate  the  cost 
of  design  alternatives.  They  provide  guidance  to 
testers  for  verification  (qualification)  of  each 
technical  requirement. 

Program-Unique  Specifications 

During  system  development  a  series  of  specifica¬ 
tions  are  generated  to  describe  the  system  at 
different  levels  of  detail.  These  program  unique 
specifications  form  the  core  of  the  configuration 
baselines.  As  shown  by  Figure  8-2,  in  addition  to 
referring  to  different  levels  within  the  system 
hierarchy,  these  baselines  are  defined  at  different 
phases  of  the  design  process. 

Initially  the  system  is  described  in  terms  of  the 
top-level  (system)  functions,  performance,  and 
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Figure  8-1.  Example  from  MIL-HDBK-881 


interfaces.  These  technical  requirements  are 
derived  from  the  operational  requirements  estab¬ 
lished  by  the  user.  This  system-level  technical 
description  is  documented  in  the  System  Specifi¬ 
cation,  which  is  the  primary  documentation  of  the 
system-level  Functional  Baseline.  The  system 
requirements  are  then  flowed  down  (allocated)  to 
the  items  below  the  system  level,  such  that  a  set  of 
design  criteria  are  established  for  each  of  those 
items.  These  item  descriptions  are  captured  in  a 
set  of  Item  Performance  Specifications,  which 
together  with  other  interface  definitions,  process 
descriptions,  and  drawings,  document  the  Allo¬ 
cated  Baseline  (sometimes  referred  to  as  the 
“Design  To”  baseline).  Having  baselined  the  design 
requirements  for  the  individual  items,  detailed 
design  follows.  Detailed  design  involves  defining 
the  system  from  top  to  bottom  in  terms  of  the 
physical  entities  that  will  be  employed  to  satisfy 
the  design  requirements.  When  detailed  design  is 
complete,  a  final  baseline  is  defined.  This  is  gen¬ 
erally  referred  to  as  the  Product  Baseline,  and, 


depending  on  the  stage  of  development,  may 
reflect  a  “Build  to”  or  “As  built”  description. 
The  Product  Baseline  is  documented  by  the  Tech¬ 
nical  Data  Package,  which  will  include  not  only 
Item  Detail  Specifications,  but  also,  Process  and 
Material  Specifications,  as  well  as  drawings, 
parts  lists,  and  other  information  that  describes 
the  final  system  in  full  physical  detail.  Figure  8- 
3  shows  how  these  specifications  relate  to  their 
associated  baselines. 

Role  of  Specifications 

Requirements  documents  express  why  the  devel¬ 
opment  is  needed.  Specification  documents  are  an 
intermediate  expression  of  what  the  needed  sys¬ 
tem  has  to  do  in  terms  of  technical  requirements 
(function,  performance,  and  interface).  Design 
documents  (drawings,  associated  lists,  etc.) 
describe  the  means  by  which  the  design  require¬ 
ments  are  to  be  satisfied.  Figure  8-4  illustrates  how 
requirements  flow  down  from  top-level  specifica- 
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System  Definition  (Functional  Baseline) 
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Qualified  System 


SI&T  =  System  Integration  and  Text 
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Figure  8-2.  Specifications  &  Levels  of  Development 


Specification 

Content 

Baseline 

System 

Spec 

Defines  mission/technical  performance  requirements. 

Allocates  requirements  to  functional  areas  and  defines  interfaces. 

Functional 

Performance 
Item  Spec 

Defines  performance  characteristics  of  CIs  and  Sis. 

Details  design  requirements  and  with  drawings  and  other 
documents  form  the  Allocated  Baseline. 

Allocated 
“Design  To” 

Detail  Item 
Spec 

Defines  form,  fit,  function,  performance,  and  test  requirements 
for  acceptance.  (Item,  process,  and  material  specs  start  the 

Product  Baseline  effort,  but  the  final  audited  baseline  includes 
all  the  items  in  the  TDP.) 

Product 
“Build  To” 

or 

“As  Built” 

Process 

Spec 

Defines  process  performed  during  fabrication. 

Material 

Spec 

Defines  production  of  raw  materials  or  semi-fabricated 
material  used  in  fabrication. 

Figure  8-3.  Specification  Types 
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tions  to  design  documentation.  Preparation  of 
specifications  are  part  of  the  system  engineering 
process,  but  also  involve  techniques  that  relate  to 
communication  skills,  both  legal  and  editorial. 
Figure  8-5  provides  some  rules-of-thumb  that  il¬ 
lustrate  this. 

In  summary,  specifications  document  what  the 
system  has  to  do,  how  well  it  has  to  do  it,  and  how 
to  verify  it  can  do  it. 

Baselines 

Baselines  formally  document  a  product  at  some 
given  level  of  design  definition.  They  are  refer¬ 
ences  for  the  subsequent  development  to  follow. 
Most  DoD  systems  are  developed  using  the  three 
classic  baselines  described  above:  functional, 
allocated,  and  product.  Though  the  program  unique 
specifications  are  the  dominant  baseline  documen¬ 
tation,  they  alone  do  not  constitute  a  baseline. 

Additional  documents  would  include  both  end  and 
enabling  product  descriptions.  End  product 
baseline  documents  would  normally  include 


those  describing  system  requirements,  functional 
architecture,  physical  architecture,  technical 
drawing  package,  and  requirements  traceability. 
Enabling  product  baseline  documents  include  a 
wide  range  of  documents  that  could  include  manu¬ 
facturing  plans  and  processes,  supportability  plan¬ 
ning,  supply  documentation,  manuals,  training 
plans  and  programs,  test  planning,  deployment 
planning,  and  others.  All  enabling  products  should 
be  reviewed  for  their  susceptibility  to  impact  from 
system  configuration  changes.  If  a  document  is 
one  that  describes  a  part  of  a  system  and  could 
require  change  if  the  configuration  changes,  then 
most  likely  it  should  be  included  as  a  baseline 
document. 

Acquisition  Program  Baselines 

Acquisition  Program  Baselines  and  Configuration 
Baselines  are  related.  To  be  accurate  the  Program 
baseline  must  reflect  the  realities  of  the  Configu¬ 
ration  Baseline,  but  the  two  should  not  be  con¬ 
fused.  Acquisition  Program  Baselines  are  high 
level  assessments  of  program  maturity  and  viabil¬ 
ity.  Configuration  Baselines  are  system  descrip¬ 
tions.  Figure  8-6  provides  additional  clarification. 


Figure  8-4.  How  Specifications  Lead  to  Design  Documents 


66 


Chapter  8 


Systems  Engineering  Process  Outputs 


•  Use  a  table  of  contents  and  define  all  abbreviations  and  acronyms. 

•  Use  active  voice. 

•  Use  “shall”  to  denote  mandatory  requirement  and  “may”  or  “should”  to  denote  guidance 
provisions. 

•  Avoid  ambiguous  provisions,  such  as  “as  necessary,”  “contractor’s  best  practice,”  “smooth 
finish,”  and  similar  terms. 

•  Use  the  System  Engineering  Process  to  identify  requirements.  Do  not  over-specify. 

•  Avoid  “tiering.”  Do  not,  in  general,  establish  requirements  in  one  document  by  referring  to 
other  documents. 

•  Only  requirement  sections  of  the  MIL-STD-961 D  formats  are  binding.  Do  not  put  requirements 
in  non-binding  sections,  such  as  Scope,  Documents,  or  Notes. 

•  Data  documentation  requirements  are  specified  in  a  Contract  Data  Requirements  List. 

Figure  &-5.  Rules-of-Thumb  for  Specification  Preparation 


Decision  Database 

The  decision  database  is  the  documentation  that 
supports  and  explains  the  configuration  solution 
decisions.  It  includes  trade  studies,  cost  effective¬ 
ness  analyses,  QFD  analysis,  models,  simulations, 
and  other  data  generated  to  understand  a  require¬ 
ment,  develop  alternative  solutions,  or  make  a 
choice  between  them.  These  items  are  retained  and 
controlled  as  part  of  the  Data  Management  pro¬ 
cess  described  in  Chapter  10. 


8.2  DOD  POLICY  AND  PRACTICE- 
SPECIFICATIONS  AND  STANDARDS 

DoD  uses  specifications  to  communicate  product 
requirements  and  standards  to  provide  guidance 
concerning  proven  methods  and  practices. 

Specifications 

DoD  uses  three  basic  classifications  of  specifica¬ 
tions:  materiel  specifications  (developed  by  DoD 
components),  Program-Unique  Specifications,  and 
non-DoD  specifications. 


•  Program  Baselines 

•  Configuration  Baselines 

-  Embody  only  the  most  important  cost, 

Identify  and  define  an  item’s  functional 

schedule,  and  performance  objectives 

and  physical  characteristics 

and  thresholds 

-  Functional  Baseline  -  Established 

-  Threshold  breach  results  in  re-evalua- 

during  PDRR 

tion  of  program  at  MDA  level 

-  Allocated  Baseline  -  Established 

-  Minimum  number  includes  key  perfor¬ 

during  EMD 

mance  parameters  in  ORD 

-  Product  Baseline  -  Established  during 

-  Specifically  evolves  over  the  develop¬ 

production,  deployment,  operations 

ment  cycle  and  is  updated  at  each  major 

and  support 

milestone  review  or  program  restruc¬ 
ture 

•  Required  on  ALL  programs  for  measuring  & 
reporting  status 

•  Documents  outputs  of  SE  Process 

Figure  8-6.  Acquisition  Program  Baselines  and  Configuration  Baselines 
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DoD  developed  specifications  describe  essen¬ 
tial  technical  requirements  for  purchase  of  ma¬ 
teriel.  Program-Unique  Specifications  are  an  in¬ 
tegral  part  of  the  system  development  process. 
Standard  practice  for  preparation  of  DoD  and 
Program-Unique  specifications  is  guided  by 
MIL-STD-961D.  This  standard  provides  guid¬ 
ance  for  the  development  of  performance  and 
detail  specifications.  MIL-STD-961D,  Appen¬ 
dix  A  provides  further  guidance  for  the  develop¬ 
ment  of  Program-Unique  Specifications. 

Non-DoD  specifications  and  standards  approved 
for  DoD  use  are  listed  in  the  DoD  Index  of  Speci¬ 
fications  and  Standards  (DoDISS). 

DoD  Policy  (Specifications) 

DoD  policy  is  to  develop  performance  specifica¬ 
tions  for  procurement  and  acquisition.  In  general, 
detail  specifications  are  left  for  contractor  devel¬ 
opment  and  use.  Use  of  a  detail  specification  in 
DoD  procurement  or  acquisition  should  be  con¬ 
sidered  only  where  absolutely  necessary,  and  then 
only  with  supporting  trade  studies  and  acquisition 
authority  approval. 

DoD  policy  gives  preference  to  the  use  of  com¬ 
mercial  solutions  to  government  requirements, 
rather  than  development  of  unique  designs.  There¬ 
fore,  the  use  of  commercial  item  specifications  and 
descriptions  should  be  a  priority  in  system  archi¬ 
tecture  development.  Only  when  no  commercial 
solution  is  available  should  government  detail 
specifications  be  employed. 

In  the  case  of  re-procurement,  where  detail  speci¬ 
fications  and  drawings  are  government  owned, 
standardization  or  interface  requirements  may 
present  a  need  for  use  of  detailed  specifications. 
Trade  studies  that  reflect  total  ownership  costs  and 
the  concerns  related  to  all  eight  primary  functions 
should  govern  decisions  concerning  the  type  of 
specification  used  for  re-procurement  of  systems, 
subsystems,  and  configuration  items.  Such  trade 
studies  and  cost  analysis  should  be  preformed  prior 
to  the  use  of  detail  specifications  or  the  decision 
to  develop  and  use  performance  specifications  in 
a  re-procurement. 


Performance  Specifications 

Performance  Specifications  state  requirements  in 
terms  of  the  required  results  with  criteria  for  veri¬ 
fying  compliance,  but  without  stating  the  meth¬ 
ods  for  achieving  the  required  results.  In  general, 
performance  specifications  define  products  in 
terms  of  functions,  performance,  and  interface 
requirements.  They  define  the  functional  require¬ 
ments  for  the  item,  the  environment  in  which  it 
must  operate,  and  interface  and  interchangeabil¬ 
ity  characteristics.  The  contractor  is  provided  the 
flexibility  to  decide  how  the  requirements  are  best 
achieved,  subject  to  the  constraints  imposed  by 
the  government,  typically  through  interface 
requirements.  System  Specifications  and  Item 
Performance  Specifications  are  examples  of 
performance  specifications. 

Detail  Specifications 

Detail  Specifications,  such  as  Item  Detail,  Mate¬ 
rial  and  Process  Specifications,  provide  design 
requirements.  This  can  include  materials  to  be 
used,  how  a  requirement  is  to  be  achieved,  or  how 
an  item  is  to  be  fabricated  or  constructed.  If  a  speci¬ 
fication  contains  both  performance  and  detail 
requirements,  it  is  considered  a  Detail  Specifica¬ 
tion,  with  the  following  exception:  Interface  and 
interchangeability  requirements  in  Performance 
Specifications  may  be  expressed  in  detailed  terms. 
For  example,  a  Performance  Specification  for 
shoes  would  specify  size  requirements  in  detailed 
terms,  but  material  or  method  of  construction 
would  be  stated  in  performance  terms. 

Software  Documentation  -  IEEE/EIA  12207 

IEEE/ELA  12207,  Software  Life  Cycle  Processes, 
describes  the  U.S.  implementation  of  the  ISO  stan¬ 
dard  on  software  processes.  This  standard  describes 
the  development  of  software  specifications  as  one 
aspect  of  the  software  development  process. 

The  process  described  in  IEEE/EIA  12207  for 
allocating  requirements  in  a  top-down  fashion  and 
documenting  the  requirements  at  all  levels  paral¬ 
lels  the  systems  engineering  process  described  in 
this  text.  The  standard  requires  first  that  system- 
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level  requirements  be  allocated  to  software  items 
(or  configuration  items)  and  that  the  software 
requirements  then  be  documented  in  terms  of 
functionality,  performance,  and  interfaces,  and  that 
qualification  requirements  be  specified.  Software 
item  requirements  must  be  traceable  to  system- 
level,  and  be  consistent  and  verifiable. 

The  developer  is  then  required  to  decompose  each 
software  item  into  software  components  and  then 
into  software  units  that  can  be  coded.  Requirements 
are  allocated  from  item  level,  to  component,  and 
finally  to  unit  level.  This  is  the  detailed  design 
activity  and  IEEE/EIA  12207  requires  that  these 
allocations  of  requirements  be  documented  in 
documents  that  are  referred  to  as  “descriptions,” 
or,  if  the  item  is  a  “stand  alone”  item,  as  “specifi¬ 
cations.”  The  content  of  these  documents  is  defined 
in  the  IEEE/EIA  standard;  however,  the  level  of 
detail  required  will  vary  by  project.  Each  project 
must  therefore  ensure  that  a  common  level  of 
expectation  is  established  among  all  stakeholders 
in  the  software  development  activity. 


Standard  Practice  for  Defense  Specifications 
-  MIL-STD-961D 

The  purpose  of  MIL-STD-961D  is  to  establish 
uniform  practices  for  specification  preparation,  to 
ensure  inclusion  of  essential  requirements,  to 
ensure  Verification  (qualification)  methods  are 
established  for  each  requirement,  and  to  aid  in  the 
use  and  analysis  of  specification  content.  MIL- 
STD-961D  establishes  the  format  and  content  of 
system,  configuration  item,  software,  process  and 
material  specifications.  These  Program-Unique 
Specifications  are  developed  through  application 
of  the  systems  engineering  process  and  represent 
a  hierarchy  as  shown  in  Figure  8-7. 

Standards 

Standards  establish  engineering  and  technical 
limitations  and  applications  for  items,  materials, 
processes,  methods,  designs,  and  engineering 
practices.  They  are  “corporate  knowledge”  docu¬ 
ments  describing  how  to  do  some  process  or  a 


Software  Product  Spec 

•  Software  Design  Description 

•  Interface  Design  Description 


Figure  8-7.  Specification  Hierarchy 
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description  of  a  body  of  knowledge.  Standards 
come  from  many  sources,  reflecting  the  practices 
or  knowledge  base  of  the  source.  Format  and 
content  of  Defense  Standards,  including  Hand¬ 
books,  are  governed  by  MIL-STD-962.  Other 
types  of  standards  in  use  in  DoD  include  Com¬ 
mercial  Standards,  Corporate  Standards,  Inter¬ 
national  Standards,  Federal  Standards,  and  Fed¬ 
eral  Information  Processing  Standards. 

DoD  Policy  (Standards) 

DoD  policy  does  not  require  standard  management 
approaches  or  manufacturing  processes  on 
contracts.  This  policy  applies  to  the  imposition  of 
both  Military  Specifications  and  Standards  and, 
in  addition,  to  the  imposition  of  Commercial  and 
Industry  Standards.  In  general,  the  preferred 
approach  is  to  allow  contractors  to  use  industry, 
government,  corporate,  or  company  standards  they 
have  determined  to  be  appropriate  to  meet 
government’s  needs.  The  government  reviews  and 
accepts  the  contractor’s  approach  through  a 
contract  selection  process  or  a  contractual  review 
process. 

The  government  should  impose  a  process  or 
standard  only  as  a  last  resort,  and  only  with  the 
support  of  an  appropriate  trade  study  analysis.  If  a 
specific  standard  is  imposed  in  a  solicitation  or 
contract,  a  waiver  will  be  required  from  an 
appropriate  Service  authority. 

However,  there  is  need  on  occasion  to  direct  the 
use  of  some  standards  for  reasons  of  standardiza¬ 
tion,  interfaces,  and  development  of  open  systems. 
A  case  in  point  is  the  mandated  use  of  the  Joint 
Technical  Architecture  (JTA)  for  defining 
interoperability  standards.  The  JTA  sets  forth  the 
set  of  interface  standards  that  are  expected  to  be 
employed  in  DoD  systems.  The  JTA  is  justifiably 
mandatory  because  it  promotes  needed  interop¬ 
erability  standardization,  establishes  supportable 
interface  standards,  and  promotes  the  development 
of  open  systems. 

DoD  technical  managers  should  be  alert  to  situa¬ 
tions  when  directed  standards  are  appropriate  to 
their  program.  Decisions  concerning  use  of 


directed  standards  should  be  confirmed  by  trade 
studies  and  requirements  traceability. 

DoD  Index  of  Specifications  and  Standards 

The  Department  of  Defense  Index  of  Specifica¬ 
tions  and  Standards  (DoDISS)  lists  all  interna¬ 
tional,  adopted  industry  standardization  documents 
authorized  for  use  by  the  military  departments, 
federal  and  military  specifications  and  standards. 
Published  in  three  volumes,  it  contains  over  30,000 
documents  in  103  Federal  Supply  Groups  broken 
down  into  850  Federal  Supply  Classes.  It  covers 
the  total  DoD  use  of  specifications  and  standards, 
ranging  from  fuel  specifications  to  international 
quality  standards. 


8.3  SUMMARY  POINTS 

•  System  Engineering  Process  Outputs  include 
the  system/configuration  item  architecture, 
specifications  and  baselines,  and  the  decision 
database. 

•  System/Configuration  Item  Architectures 
include  the  physical  architecture  and  the 
associated  products  and  services. 

•  Program-Unique  specifications  are  a  primary 
output  of  the  System  Engineering  Process. 
Program-Unique  specifications  describe  what 
the  system  or  configuration  item  must  accom¬ 
plish  and  how  it  will  be  verified.  Program- 
Unique  specifications  include  the  System,  Item 
Performance,  and  Item  Detail  Specifications. 
The  System  Specification  describes  the  system 
requirements,  while  Item  Performance  and  Item 
Detail  Specifications  describe  configuration 
item  requirements. 

•  Configuration  baselines  are  used  to  manage  and 
control  the  technical  development.  Program 
baselines  are  used  for  measuring  and  supporting 
program  status. 

•  The  Decision  Database  includes  those  docu¬ 
ments  or  software  that  support  understanding 
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and  decision  making  during  formulation  of 
the  configuration  baselines. 

•  DoD  policy  is  to  develop  performance  speci¬ 
fications  for  procurement  and  acquisition.  Use 
of  other  than  performance  specifications  in  a 
contract  must  be  justified  and  approved. 


•  It  is  DoD  policy  not  to  require  standard 
management  approaches  or  manufacturing 
processes  on  contracts. 

•  Mandatory  use  of  some  standard  practices  are 
necessary,  but  must  be  justified  through  analy¬ 
sis.  A  case  in  point  is  the  mandatory  use  of  the 
standards  listed  in  the  JTA. 
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WORK  BREAKDOWN 
STRUCTURE 


9.1  INTRODUCTION 

The  Work  Breakdown  Structure  (WBS)  is  a  means 
of  organizing  system  development  activities  based 
on  system  and  product  decompositions.  The  sys¬ 
tems  engineering  process  described  in  earlier  chap¬ 
ters  produces  system  and  product  descriptions. 
These  product  architectures,  together  with  associ¬ 
ated  services  (e.g.,  program  management,  systems 
engineering,  etc.)  are  organized  and  depicted  in  a 
hierarchical  tree-like  structure  that  is  the  Work 
Breakdown  Structure. 

Because  the  WBS  is  a  direct  derivative  of  the  physi¬ 
cal  and  systems  architectures  it  could  be  consid¬ 
ered  an  output  of  the  systems  engineering  process. 
It  is  being  presented  here  as  a  Systems  Analysis 
and  Control  tool  because  of  its  essential  utility  for 
all  aspects  of  the  systems  engineering  process.  It 


is  used  to  structure  development  activities,  to  iden¬ 
tify  data  and  documents,  and  to  organize  integrated 
teams,  and  for  other  non-technical  program 
management  purposes. 

WBS  Role  in  DoD  Systems  Engineering 

DoD  5000.2-R  requires  that  a  program  WBS  be 
established  to  provide  a  framework  for  program 
and  technical  planning,  cost  estimating,  resource 
allocation,  performance  measurement,  and  status 
reporting.  The  WBS  is  used  to  define  the  total 
system,  to  display  it  as  a  product-oriented  family 
tree  composed  of  hardware,  software,  services, 
data,  and  facilities,  and  to  relate  these  elements  to 
each  other  and  to  the  end  product.  Program  offices 
are  to  tailor  a  program  WBS  using  the  guidance 
provided  in  MIL-HDBK-881. 


Figure  9-1 .  Architecture  to  WBS  Flow 
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The  program  WBS  is  developed  initially  to  define 
the  top  three  levels.  As  the  program  proceeds 
through  development  and  is  further  defined,  pro¬ 
gram  managers  should  ensure  that  the  WBS  is 
extended  to  identify  all  high-cost  and  high-risk 
elements  for  management  and  reporting,  while 
ensuring  the  contractor  has  complete  flexibility  to 
extend  the  WBS  below  the  reporting  requirement 
to  reflect  how  work  will  be  accomplished. 

Basic  Purposes  of  the  WBS 

Organizational: 

The  WBS  provides  a  coordinated,  complete,  and 
comprehensive  view  of  program  management.  It 
establishes  a  structure  for  organizing  system 
development  activities,  including  IPT  design, 
development,  and  maintenance. 

Business: 

It  provides  a  structure  for  budgets  and  cost  esti¬ 
mates.  It  is  used  to  organize  collection  and  analy¬ 
sis  of  detailed  costs  for  earned  value  reports  (Cost 
Performance  Reports  or  Cost/Schedule  Control 
System  Criteria  reporting). 

Technical: 

The  WBS  establishes  a  structure  for: 

•  Identifying  products,  processes,  and  data. 


•  Organizing  risk  management  analysis  and 
tracking. 

•  Enabling  configuration  and  data  management. 
It  helps  establish  interface  identification  and 
control. 

•  Developing  work  packages  for  work  orders  and 
material/part  ordering. 

•  Organizing  technical  reviews  and  audits. 

The  WBS  is  used  to  group  product  items  for  speci¬ 
fication  development,  to  develop  Statements  of 
Work,  and  to  identify  specific  contract  deliverables. 

WBS  -  Benefits 

The  WBS  allows  the  total  system  to  be  described 
through  a  logical  breakout  of  product  elements  into 
work  packages.  A  WBS,  correctly  prepared,  will 
account  for  all  program  activity.  It  links  program 
objectives  and  activities  with  resources,  facilitates 
initial  budgets,  and  simplifies  subsequent  cost 
reporting.  The  WBS  allows  comparison  of  vari¬ 
ous  independent  metrics  and  other  data  to  look  for 
comprehensive  trends. 

It  is  a  foundation  for  all  program  activities, 
including  program  and  technical  planning,  event 


Figure  9-2.  Program  WBS  -  The  Product  Part  (Physical  Architecture) 
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schedule  definition,  configuration  management, 
risk  management,  data  management,  specification 
preparation,  Statement  of  Work  preparation,  status 
reporting  and  problem  analysis,  cost  estimates,  and 
budget  formulation. 

9.2  WBS  DEVELOPMENT 

The  physical  and  system  architectures  are  used  to 
prepare  the  WBS.  The  architectures  should  be 
reviewed  to  ensure  that  all  necessary  products  and 
services  are  identified,  and  that  the  top-down  struc¬ 
ture  provides  a  continuity  of  flow  down  for  all 
tasks.  Enough  levels  must  be  provided  to  identify 
work  packages  for  cost/schedule  control  purposes. 
If  too  few  levels  are  identified,  then  management 
visibility  and  integration  of  work  packages  may 
suffer.  If  too  many  levels  are  identified,  then  pro¬ 
gram  review  and  control  actions  may  become 
excessively  time-consuming. 


The  first  three  Work  Breakdown  Structure  Levels 
are  organized  as: 

Level  1  -  Overall  System 

Level  2  -  Major  Element  (Segment) 

Level  3  -  Subordinate  Components  (Prime 
Items) 

Levels  below  the  first  three  represent  component 
decomposition  down  to  the  configuration  item 
level.  In  general,  the  government  is  responsible 
for  the  development  of  the  first  three  levels,  and 
the  contractor(s)  for  levels  below  three. 

DoD  Practice 

In  accordance  with  DoD  Instruction  5000.2-R 
direction  and  common  DoD  practice  as  established 
in  MIL-HDBK-881,  the  program  office  develops 
a  program  WBS  and  a  contract  WBS  for  each  con¬ 
tract.  The  program  WBS  is  the  WBS  that  repre¬ 
sents  the  total  system,  i.e.,  the  WBS  that  describes 
the  system  architecture.  The  contract  WBS  is  the 
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Figure  9-3.  The  Complete  Work  Breakdown  Structure 
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Figure  9-4.  Contract  WBS 


part  of  the  program  WBS  that  relates  to 
deliverables  and  tasks  of  a  specific  contract. 

MIL-HDBK-881  is  used  by  the  program  office  to 
support  the  systems  engineering  process  in  devel¬ 
oping  the  first  three  levels  of  the  program  WBS, 
and  to  provide  contractors  with  guidance  for  lower 
level  WBS  development.  As  with  most  standards 
and  handbooks,  use  of  MIL-HDBK-881  cannot  be 
specified  as  a  contract  requirement. 

Though  WBS  development  is  a  systems  engineer¬ 
ing  activity,  it  impacts  cost  and  budget  profession¬ 
als,  as  well  as  contracting  officers.  An  integrated 
team  representing  these  stakeholders  should  be 
formed  to  support  WBS  development. 

WBS  Anatomy 

A  program  WBS  has  an  end  product  part  and  an 
enabling  product  part.  The  end  product  part  of  the 
system  typically  consists  of  the  prime  mission 
product(s)  delivered  to  the  operational  customer. 
This  part  of  the  WBS  is  based  on  the  physical 
architectures  developed  from  operational  require¬ 
ments.  It  represents  that  part  of  the  WBS  involved 
in  product  development.  Figure  9-2  presents  a 
simple  example  of  a  program  WBS  product  part. 


The  “enabling  product”  part  of  the  system  in¬ 
cludes  the  products  and  services  required  to  de¬ 
velop,  produce,  and  support  the  end  product(s). 
This  part  of  the  WBS  includes  the  horizontal 
elements  of  the  system  architecture  (exclusive 
of  the  end  products),  and  identifies  all  the  prod¬ 
ucts  and  services  necessary  to  support  the  life 
cycle  needs  of  the  product.  Figure  9-3  shows  an 
example  of  the  top  three  levels  of  a  complete 
WBS  tree. 

Contract  WBS 

A  contract  WBS  is  developed  by  the  program  office 
in  preparation  for  contracting  for  work  required  to 
develop  the  system.  It  is  further  developed  by  the 
contractor  after  contract  award.  The  contract  WBS 
is  that  portion  of  the  program  WBS  that  is  specifi¬ 
cally  being  tasked  through  the  contract.  A  simple 
example  of  a  contract  WBS  derived  from  the 
program  WBS  shown  in  Figure  9-2  is  provided  by 
Figure  9-4.  Figure  9-4,  like  Figure  9-2,  only 
includes  the  product  part  of  the  contract  WBS.  A 
complete  contract  WBS  would  include  associated 
enabling  products,  similar  to  those  identified  in 
Figure  9-3.  The  resulting  complete  contract  WBS 
is  used  to  organize  and  identify  contractor  tasks. 
The  program  office’s  preliminary  version  is  used 
to  develop  a  Statement  of  Work  for  the  Request 
for  Proposals. 
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9.3  DESIGNING  AND  TRACKING  WORK 

A  prime  use  of  the  WBS  is  the  design  and  track¬ 
ing  of  work.  The  WBS  is  used  to  establish  what 
work  is  necessary,  a  logical  decomposition  down 
to  work  packages,  and  a  method  for  organizing 
feedback.  As  shown  by  Figure  9-5,  the  WBS  ele¬ 
ment  is  matrixed  against  those  organizations  in 
the  company  responsible  for  the  task.  This  creates 
cost  accounts  and  task  definition  at  a  detailed  level. 
It  allows  rational  organization  of  integrated  teams 
and  other  organizational  structures  by  helping 
establish  what  expertise  and  functional  support  is 
required  for  a  specific  WBS  element.  It  further 
allows  precise  tracking  of  technical  and  other 
management. 


WBS  Dictionary 

As  part  of  the  work  and  cost  control  use  of  the 
WBS  a  Work  Breakdown  Dictionary  is  developed. 
For  each  WBS  element  a  dictionary  entry  is  pre¬ 
pared  that  describes  the  task,  what  costs  (activi¬ 
ties)  apply,  and  the  references  to  the  associated 
Contract  Line  Item  Numbers  and  Statement  of 
Work  paragraph.  An  example  of  a  level  2  WBS 
element  dictionary  entry  is  shown  as  Figure  9-6. 


9.3  SUMMARY  POINTS 

•  The  WBS  is  an  essential  tool  for  the  organiza¬ 
tion  and  coordination  of  systems  engineering 
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processes,  and  it  is  a  product  of  the  systems 
engineering  process. 

•  Its  importance  extends  beyond  the  technical 
community  to  business  professionals  and  con¬ 
tracting  officers.  The  needs  of  all  stakeholders 
must  be  considered  in  its  development.  The  pro¬ 
gram  office  develops  the  program  WBS  and 
a  high-level  contract  WBS  for  each  contract. 
The  contractors  develop  the  lower  levels  of 


the  contract  WBS  associated  with  their  con¬ 
tract. 

•  The  system  architecture  provides  the  structure 
for  a  program  WBS.  Statement  of  Work  tasks 
flow  from  this  WBS. 

•  The  WBS  provides  a  structure  for  organizing 
IPTs  and  tracking  metrics. 


Index  Item  No.  2 

WBS  Level  2 

CONTRACT  NUMBER 

F33657-72-C-0923 

WBS  Element 

WBS  Title 

Contract 

A10100 

Air  Vehicle 

Line  Item: 

Date  Revision  No. 

Revision  Auth 

Approved  Chg 

0001, 0001 AA,  0001 AB,  0001  AC,  0001  AD 

0001 AE,  0001 AF,  0001AG,  0001  AH 

Specification  No. 

Specification  Title: 

Prime  Item  Development 

689E078780028 

Specificaiton  for  AGM  86A  Air  Vehicle/ 

Airframe 

Element  Task  Description 
Technical  Content: 

The  Air  Vehicle  element  task  description  refers  to  the  effort 
required  to  develop,  fabricate,  integrate  and  test  the 
airframe  segment,  portions  of  the  Navigation/Guidance 
element,  and  Airborne  Development  Test  Equipment  and 
Airborne  Operational  Test  Equipment  and  to  the  integration 
assembly  and  check-out  of  these  complete  elements, 
together  with  the  Engine  Segment,  to  produce  the  complete 
Air  Vehicle.  The  lower-level  elements  included  and 
summarized  in  the  Air  Vehicle  element  are: 

Airframe  Segment  (A1 1100),  Navigation/Guidance 
Segment  (A32100),  Airborne  Development  Test 
Equipment  (A61 1 00),  and  Airborne  Operational  Test 
Equipment  (A61 200) 


Cost  Description 

MPC/PMC  Work  Order/Work  Auth 

A1 01 00  See  lower  level 

WBS  Elements 

Cost  Content  -  System  Contractor 

The  cost  to  be  accumulated  against  this  element  includes 
a  summarization  of  all  costs  required  to  plan,  develop, 
fabricate,  assemble,  integrate  and  perform  development 
testing,  analysis  and  reporting  for  the  air  vehicle.  It  also 
includes  all  costs  associated  with  the  required  efforts  in 
integrating,  assembling  and  checking  our  GFP  required  to 
create  this  element. 

Applicable  SOW  Paragraph 

3.6.2 


Figure  9-6.  Work  Breakdown  Structure  Dictionary 
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10.1  FOUNDATIONS 
Configuration  Defined 

A  “configuration”  consists  of  the  functional,  physi¬ 
cal,  and  interface  characteristics  of  existing  or 
planned  hardware,  firmware,  software  or  a  com¬ 
bination  thereof  as  set  forth  in  technical  documen¬ 
tation  and  ultimately  achieved  in  a  product.  The 
configuration  is  formally  expressed  in  relation  to 
a  Functional,  Allocated,  or  Product  configuration 
baseline  as  described  in  Chapter  8. 

Configuration  Management 

Configuration  management  permits  the  orderly 
development  of  a  system,  subsystem,  or  configu¬ 
ration  Item.  A  good  configuration  management 
program  ensures  that  designs  are  traceable  to 
requirements,  that  change  is  controlled  and 
documented,  that  interfaces  are  defined  and 
understood,  and  that  there  is  consistency  between 
the  product  and  its  supporting  documentation. 
Configuration  management  provides  documenta¬ 
tion  that  describes  what  is  supposed  to  be  pro¬ 
duced,  what  is  being  produced,  what  has  been 
produced,  and  what  modifications  have  been  made 
to  what  was  produced. 

Configuration  management  is  supported  and 
performed  by  integrated  teams  in  an  Integrated 
Product  and  Process  Development  (IPPD)  envi¬ 
ronment.  Configuration  management  is  closely 
associated  with  technical  data  management  and 
interface  management.  Data  and  interface  manage¬ 
ment  is  essential  for  proper  configuration 
management,  and  the  configuration  management 
effort  has  to  include  them. 


DoD  Application  of 
Configuration  Management 

During  the  development  contract,  the  Government 
should  maintain  configuration  control  of  the 
functional  and  performance  requirements  only, 
giving  contractors  responsibility  for  the  detailed 
design.  (SECDEF  Memo  of  29  Jun  94.)  This  implies 
government  control  of  the  Functional  (system 
requirements)  Baseline.  Decisions  regarding  whether 
or  not  the  government  will  take  control  of  the 
lower-level  baselines  (allocated  and  product 
baselines),  and  when  dependent  on  the  require¬ 
ments  and  strategies  are  needed  for  the  particular 
program.  In  general,  government  control  of  lower- 
level  baselines,  if  exercised,  will  take  place  late  in 
the  development  program  after  design  has  stabilized. 

Configuration  Management  Planning 

Planning  a  configuration  management  effort  should 
consider  the  basics:  what  has  to  be  done,  how 
should  it  be  done,  who  should  do  it,  when  should 
it  be  done,  and  what  resources  are  required.  Plan¬ 
ning  should  include  the  organizational  and  func¬ 
tional  structure  that  will  define  the  methods  and 
procedures  to  manage  system  or  component  func¬ 
tional  and  physical  characteristics,  interfaces,  and 
documents.  It  should  also  include  statements  of 
responsibility  and  authority,  methods  of  control, 
methods  of  audit  or  verification,  milestones,  and 
schedules.  EIA IS-649,  National  Consensus  Stan¬ 
dard  for  Configuration  Management,  and  Mil- 
HDBK-61  can  be  used  as  planning  guidance. 

Configuration  Item  (Cl) 

A  key  concept  that  affects  planning  is  the  con¬ 
figuration  item  (Cl).  Cl  decisions  will  determine 
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what  configurations  will  be  managed.  CIs  are  an 
aggregation  of  hardware,  firmware,  or  computer 
software,  or  any  of  their  discrete  portions,  which 
satisfies  an  end-use  function  and  is  designated  for 
separate  configuration  management.  Any  item 
required  for  logistic  support  and  designated  for 
separate  procurement  is  generally  identified  as  CI. 
Components  can  be  designated  CIs  because  of 
crucial  interfaces  or  need  to  be  integrated  with 
operation  with  other  components  within  or  out¬ 
side  of  the  system.  An  item  can  be  designated  CI 
if  it  is  developed  wholly  or  partially  with  govern¬ 
ment  funds,  including  non-developmental  items 
(NDI)  if  additional  development  of  technical  data 
is  required.  All  CIs  are  directly  traceable  to  the 
work  breakdown  structure. 

Impact  of  CI  Designation 

CI  designation  requires  a  separate  configuration 
management  effort  for  the  CI,  or  groupings  of 
related  CIs.  The  decision  to  place  an  item,  or  items, 
under  formal  configuration  control  results  in; 

•  Separate  specifications, 

•  Formal  approval  of  changes, 

•  Discrete  records  for  configuration  status 
accounting, 

•  Individual  design  reviews  and  configuration 
audits, 

•  Discrete  identifiers  and  name  plates, 

•  Separate  qualification  testing,  and 

•  Separate  operating  and  user  manuals. 

10.2  CONFIGURATION  MANAGEMENT 
STRUCTURE 

Configuration  management  comprises  four 
interrelated  efforts; 

•  Identification, 


•  Control, 

•  Status  Accounting,  and 

•  Audits. 

Also  directly  associated  with  configuration  man¬ 
agement  are  data  management  and  interface 
management.  Any  configuration  management 
planning  effort  must  consider  all  six  elements. 

Identification 

Configuration  Identification  consists  of  docu¬ 
mentation  of  formally  approved  baselines  and 
specifications,  including: 

•  Selection  of  the  configuration  items  (CI), 

•  Determination  of  the  types  of  configuration 
documentation  required  for  each  CI, 

•  Documenting  the  functional  and  physical 
characteristics  of  each  CI, 

•  Establishing  interface  management  procedures, 
organization,  and  documentation, 

•  Issuance  of  numbers  and  other  identifiers 
associated  with  the  system/CI  configuration 
structure,  including  internal  and  external 
interfaces,  and 

•  Distribution  of  CI  identification  and  related 
configuration  documentation. 

Configuration  Documentation 

Configuration  documentation  is  technical  docu¬ 
mentation  that  identifies  and  defines  the  item’s 
functional  and  physical  characteristics.  It  is 
developed,  approved,  and  maintained  through  three 
distinct  evolutionary  increasing  levels  of  detail.  The 
three  levels  of  configuration  documentation  form 
the  three  baselines  and  are  referred  to  as  functional, 
allocated,  and  product  configuration  documenta¬ 
tion.  These  provide  the  specific  technical  descrip¬ 
tion  of  a  system  or  its  components  at  any  point  in 
time. 
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Control 

Configuration  Control  is  the  systematic  proposal, 
justification,  prioritization,  evaluation,  coordina¬ 
tion,  approval  or  disapproval,  and  implementation 
of  all  approved  changes  in  the  configuration  of  a 
system/CI  after  formal  establishment  of  its 
baseline.  In  other  words,  it  is  how  a  system  (and 
its  configuration  items)  change  control  process  is 
executed  and  managed. 

Configuration  Control  provides  management 
visibility,  ensures  all  factors  associated  with  a 
proposed  change  are  evaluated,  prevents  unneces¬ 
sary  or  marginal  changes,  and  establishes  change 
priorities.  In  DoD  it  consists  primarily  of  a  change 
process  that  formalizes  documentation  and 
provides  a  management  structure  for  change 
approval. 

Change  Documents  Used  for 
Government  Controlled  Baselines 

There  are  three  types  of  change  documents  used 
to  control  baselines  associated  with  government 
configuration  management:  Engineering  Change 
Proposal,  Request  for  Deviation,  and  Request  for 
Waivers. 

•  Engineering  Change  Proposals  (ECP)  identify 
need  for  a  permanent  configuration  change. 
Upon  approval  of  an  ECP  a  new  configuration 
is  established. 

•  Requests  for  Deviation  or  Waiver  propose  a 
temporary  departure  from  the  baseline.  They 
allow  for  acceptance  of  non-conforming 
material.  After  acceptance  of  a  deviation  or 
waiver  the  documented  configuration  remains 
unchanged. 

Engineering  Change  Proposal  (ECP) 

An  ECP  is  documentation  that  describes  and 
suggests  a  change  to  a  configuration  baseline. 
Separate  ECPs  are  submitted  for  each  change  that 
has  a  distinct  objective.  To  provide  advanced  notice 
and  reduce  paperwork,  Preliminary  ECPs  or 
Advance  Change/Study  Notices  can  be  used 


preparatory  to  issue  of  a  formal  ECP.  Time  and 
effort  for  the  approval  process  can  be  further 
reduced  through  use  of  joint  government  and 
contractor  integrated  teams  to  review  and  edit 
preliminary  change  proposals. 

ECPs  are  identified  as  Class  I  or  Class  II.  Class  I 
changes  require  government  approval  before 
changing  the  configuration.  These  changes  can 
result  from  problems  with  the  baseline  require¬ 
ment,  safety,  interfaces,  operating/servicing  capa¬ 
bility,  preset  adjustments,  human  interface  includ¬ 
ing  skill  level,  or  training.  Class  I  changes  can  also 
be  used  to  upgrade  already  delivered  systems  to 
the  new  configuration  through  use  of  retrofit,  mod 
kits,  and  the  like.  Class  I  ECPs  are  also  used  to 
change  contractual  provisions  that  do  not  directly 
impact  the  configuration  baseline;  for  example, 
changes  affecting  cost,  warranties,  deliveries,  or 
data  requirements.  Class  I  ECPs  require  program 
office  approval,  which  is  usually  handled  through 
a  formal  Configuration  Control  Board,  chaired  by 
the  government  program  manager  or  delegated 
representative. 

Class  II  changes  correct  minor  conflicts,  typos, 
and  other  “housekeeping”  changes  that  basically 
corrects  the  documentation  to  reflect  the  current 
configuration.  Class  II  applies  only  if  the  configu¬ 
ration  is  not  changed  when  the  documentation  is 
changed.  Class  II  ECPs  are  usually  handled  by 
the  in-plant  government  representative.  Class  II 
ECPs  generally  require  only  that  the  government 
concurs  that  the  change  is  properly  classified. 
Under  an  initiative  by  the  Defense  Contract 
Management  Command  (DCMC),  contractors  are 
increasingly  delegated  the  authority  to  make  ECP 
classification  decisions. 

Figure  10-1  shows  the  key  attributes  associated 
with  ECPs.  The  preliminary  ECP,  mentioned  in 
Figure  10-1,  is  a  simplified  version  of  a  formal 
ECP  that  explains  the  proposed  ECP,  and  estab¬ 
lishes  an  approximate  schedule  and  cost  for  the 
change.  The  expense  of  an  ECP  development  is 
avoided  if  review  of  the  Preliminary  ECP  indicates 
the  change  is  not  viable.  The  approach  used  for 
preliminary  ECPs  vary  in  their  form  and  name. 
Both  Preliminary  ECPs  and  Advanced  Change/ 
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Figure  10-1.  ECP  Designators 


Study  Notices  have  been  used  to  formalize  this 
process,  but  forms  tailored  to  specific  programs 
have  also  been  used. 

Configuration  Control  Board  (CCB) 

A  CCB  is  formed  to  review  Class  I  ECPs  for 
approval,  and  make  a  recommendation  to  approve 
or  not  approve  the  proposed  change.  The  CCB 
chair,  usually  the  Program  Manager,  makes  the 
final  decision.  Members  advise  and  recommend, 
but  the  authority  for  the  decision  rests  with  the 
chair.  CCB  membership  should  represent  the  eight 
primary  functions  with  the  addition  of  representa¬ 
tion  of  the  procurement  office,  program  control 
(budget),  and  Configuration  Control  manager,  who 
serves  as  the  CCB  secretariat. 

The  CCB  process  is  shown  in  Figure  10-2.  The 
process  starts  with  the  contractor.  A  request  to  the 
contractor  for  an  ECP  or  Preliminary  ECP  is 
necessary  to  initiate  a  government  identified 
configuration  change.  The  secretariat’s  review  pro¬ 
cess  includes  assuring  appropriate  government 
contractual  and  engineering  review  is  done  prior 
to  receipt  by  the  CCB. 


CCB  Management  Philosophy 

The  CCB  process  is  a  configuration  control  pro¬ 
cess,  but  it  is  also  a  contractual  control  process. 
Decisions  made  by  the  CCB  chair  affects  the  con¬ 
tractual  agreement  and  program  baseline  as  well 
as  the  configuration  baseline.  Concerns  over  con¬ 
tractual  policy,  program  schedule,  and  budget  can 
easily  come  into  conflict  with  concerns  relating  to 
configuration  management,  technical  issues,  and 
technical  activity  scheduling.  The  CCB  technical 
membership  and  CCB  secretariat  is  responsible 
to  provide  a  clear  view  of  the  technical  need  and 
the  impact  of  alternate  solutions  to  these  conflicts. 
The  CCB  secretariat  is  further  responsible  to  see 
that  the  CCB  is  fully  informed  and  prepared, 
including  assuring  that: 

•  A  government/contractor  engineering  work¬ 
ing  group  has  analyzed  the  ECP  and  supporting 
data,  prepared  comments  for  CCB  consideration, 
and  is  available  to  support  the  CCB; 

•  All  pertinent  information  is  available  for  review; 

•  The  ECP  has  been  reviewed  by  appropriate 
functional  activities;  and 
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Figure  10-2.  Configuration  Control  Board 


•  Issues  have  been  identified  and  addressed. 

CCB  Documentation 

Once  the  CCB  chair  makes  a  decision  concerning 
an  ECP,  the  CCB  issues  a  Configuration  Control 
Board  Directive  that  distributes  the  decision  and 
identifies  key  information  relating  to  the 
implementation  of  the  change: 

•  Implementation  plan  (who  does  what  when); 

•  Contracts  affected  (prime  and  secondary); 

•  Dates  of  incorporation  into  contracts; 

•  Documentation  affected  (drawings,  specifica¬ 
tions,  technical  manuals,  etc.),  associated  cost, 
and  schedule  completion  date;  and 

•  Identification  of  any  orders  or  directives  needed 
to  be  drafted  and  issued. 


Request  for  Deviation  or  Waiver 

A  deviation  is  a  specific  written  authorization, 
granted  prior  to  manufacture  of  an  item,  to  depart 
from  a  performance  or  design  requirement  for  a 
specific  number  of  units  or  a  specific  period  of  time. 

A  waiver  is  a  written  authorization  to  accept  a  con¬ 
figuration  item  that  departs  from  specified  require¬ 
ments,  but  is  suitable  for  use  “as  is”  or  after  repair. 

Requests  for  deviation  and  waivers  relate  to  a  tem¬ 
porary  baseline  departure  that  can  affect  system 
design  and/or  performance.  The  baseline  remains 
unchanged  and  the  government  makes  a  determi¬ 
nation  whether  the  alternative  “non-conforming” 
configuration  results  in  an  acceptable  substitute. 
Acceptable  substitute  usually  implies  that  there 
will  be  no  impact  on  support  elements,  systems 
affected  can  operate  effectively,  and  no  follow-up 
or  correction  is  required.  The  Federal  Acquisition 
Regulations  (FAR)  requires  “consideration”  on 
government  contracts  when  the  Government 
accepts  a  “non-conforming”  unit. 
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The  distinction  between  Request  for  Deviation  and 
Request  for  a  Waiver  is  that  a  deviation  is  used 
before  final  assembly  of  affected  unit,  and  a  waiver 
is  used  after  final  assembly  or  acceptance  testing 
of  affected  unit. 

Status  Accounting 

Configuration  Status  Accounting  is  the  recording 
and  reporting  of  the  information  that  is  needed  to 
manage  the  configuration  effectively,  including: 

•  A  listing  of  the  approved  configuration 
documentation, 

•  The  status  of  proposed  changes,  waivers  and 
deviations  to  the  configuration  identification, 

•  The  implementation  status  of  approved 
changes,  and 

•  The  configuration  of  all  units,  including  those 
in  the  operational  inventory. 

Purpose  of  Configuration  Status  Accounting 

Configuration  Status  Accounting  provides  infor¬ 
mation  required  for  configuration  management  by: 

•  Collecting  and  recording  data  concerning: 

-  Baseline  configurations, 

-  Proposed  changes,  and 

-  Approved  changes. 

•  Disseminating  information  concerning: 

-  Approved  configurations, 

-  Status  and  impact  of  proposed  changes, 

-  Requirements,  schedules,  impact  and 
status  of  approved  changes,  and 

-  Current  configurations  of  delivered  items. 

Audits 

Configuration  Audits  are  used  to  verify  a  system 
and  its  components’  conformance  to  their  configu¬ 
ration  documentation.  Audits  are  key  milestones 
in  the  development  of  the  system  and  do  not  stand 
alone.  The  next  chapter  will  show  how  they  fit  in 
the  overall  process  of  assessing  design  maturity. 


Functional  Configuration  Audits  (FCA)  and  the 
System  Verification  Review  (SVR)  are  performed 
in  the  Engineering  and  Manufacturing  Develop¬ 
ment  Phase.  FCA  is  used  to  verify  that  actual 
performance  of  the  configuration  item  meets 
specification  requirements.  The  System  Verifica¬ 
tion  Review  (SVR)  serves  as  system  level  audit 
after  FCAs  have  been  conducted. 

The  Physical  Configuration  Audit  (PCA)  is  held 
during  Production,  Fielding,  and  Operational  Sup¬ 
port  as  a  formal  examination  of  a  production 
representative  unit  against  the  draft  technical  data 
package  (product  baseline  documentation). 

Most  audits,  whether  FCA  or  PCA,  are  today  ap¬ 
proached  as  a  series  of  “rolling”  reviews  in  which 
items  are  progressively  audited  as  they  are  pro¬ 
duced  such  that  the  final  FCA  or  PCA  becomes 
significantly  less  oppressive  and  disruptive  to  the 
normal  flow  of  program  development. 

10.3  INTERFACE  MANAGEMENT 

Interface  Management  consists  of  identifying  the 
interfaces,  establishing  working  groups  to  man¬ 
age  the  interfaces,  and  the  group’s  development 
of  interface  control  documentation.  Interface  Man¬ 
agement  identifies,  develops,  and  maintains  the 
external  and  internal  interfaces  necessary  for  sys¬ 
tem  operation.  It  supports  the  configuration  man¬ 
agement  effort  by  ensuring  that  configuration 
decisions  are  made  with  full  understanding  of  their 
impact  outside  of  the  area  of  the  change. 

Interface  Identification 

An  interface  is  a  functional,  physical,  electrical, 
electronic,  mechanical,  hydraulic,  pneumatic, 
optical,  software,  or  similar  characteristic  required 
to  exist  at  a  common  boundary  between  two  or 
more  systems,  products,  or  components.  Normally, 
in  a  contractual  relationship  the  procuring  agency 
identifies  external  interfaces,  sets  requirements  for 
integrated  teams,  and  provides  appropriate  person¬ 
nel  for  the  teams.  The  contracted  design  agent  or 
manufacturer  manages  internal  interfaces;  plans, 
organizes,  and  leads  design  integrated  teams; 
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maintains  internal  and  external  interface  re¬ 
quirements;  and  controls  interfaces  to  ensure 
accountability  and  timely  dissemination  of 
changes. 

Interface  Control  Working  Group  (ICWG) 

The  ICWG  is  the  traditional  forum  to  establish 
official  communications  link  between  those 
responsible  for  the  design  of  interfacing  systems 
or  components.  Within  the  IPPD  framework 
ICWGs  can  be  integrated  teams  that  establish  link¬ 
age  between  interfacing  design  IPTs,  or  could  be 
integrated  into  an  system  level  engineering  work¬ 
ing  group.  Membership  of  ICWGs  or  comparable 
integrated  teams  should  include  membership  from 
each  contractor,  significant  vendors,  and  partici¬ 
pating  government  agencies.  The  procuring 
program  office  (external  and  selected  top-level 
interfaces)  or  prime  contractor  (internal  interfaces) 
generally  designates  the  chair. 

Interface  Control  Documentation  (ICD) 

Interface  Control  Documentation  includes  Inter¬ 
face  Control  Drawings,  Interface  Requirements 
Specifications,  and  other  documentation  that 
depicts  physical  and  functional  interfaces  of  related 
or  co-functioning  systems  or  components.  ICD  is 
the  product  of  ICWGs  or  comparable  integrated 
teams,  and  their  purpose  is  to  establish  and  main¬ 
tain  compatibility  between  interfacing  systems  or 
components. 

Open  Systems  Interface  Standards 

To  minimize  the  impact  of  unique  interface 
designs,  improve  interoperability,  maximize  the 
use  of  commercial  components,  and  improve  the 
capacity  for  future  upgrade,  an  open  systems 
approach  should  be  a  significant  part  of  interface 
control  planning.  The  open  systems  approach  in¬ 
volves  selecting  industry-recognized  specifications 
and  standards  to  define  system  internal  and  exter¬ 
nal  interfaces.  An  open  system  is  characterized  by: 

•  Increased  use  of  functional  partitioning  and 
modular  design  to  enhance  flexibility  of 
component  choices  without  impact  on  interfaces, 


•  Use  of  well-defined,  widely  used,  non-propri¬ 
etary  interfaces  or  protocols  based  on  standards 
developed  or  adopted  by  industry  recognized 
standards  institutions  or  professional  societies, 
and 

•  Explicit  provision  for  expansion  or  upgrading 
through  the  incorporation  of  additional  or 
higher  performance  elements  with  minimal 
impact  on  the  system. 

DoD  mandatory  guidance  for  information  technol¬ 
ogy  standards  is  in  the  Joint  Technical  Architecture. 

10.4  DATA  MANAGEMENT 

Data  management  documents  and  maintains  the 
database  reflecting  system  life  cycle  decisions, 
methods,  feedback,  metrics,  and  configuration 
control.  It  directly  supports  the  configuration  status 
accounting  process.  Data  Management  governs  and 
controls  the  selection,  generation,  preparation, 
acquisition,  and  use  of  data  imposed  on  contractors. 

Data  Required  By  Contract 

Data  is  defined  as  recorded  information,  regard¬ 
less  of  form  or  characteristic,  and  includes  all  the 
administrative,  management,  financial,  scientific, 
engineering,  and  logistics  information  and  docu¬ 
mentation  required  for  delivery  from  the  contrac¬ 
tor.  Contractually  required  data  is  classified  as  one 
of  three  types: 

•  Type  I:  Technical  data 

•  Type  II:  Non-technical  data 

•  Type  III:  One-time  use  data  (technical  or  non¬ 
technical) 

Data  is  acquired  for  two  basic  purposes: 

•  Information  feedback  from  the  contractor  for 
program  management  control,  and 

•  Decision  making  information  needed  to 
manage,  operate,  and  support  the  system  (e.g., 
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specifications,  technical  manuals,  engineering 
drawings,  etc.). 

Data  analysis  and  management  is  expensive  and 
time  consuming.  Present  DoD  philosophy  requires 
that  the  contractor  manage  and  maintain  signifi¬ 
cant  portions  of  the  technical  data,  including  the 
Technical  Data  Package  (TDP).  Note  that  this  does 
not  mean  the  government  isn’t  paying  for  its 
development  or  shouldn’t  receive  a  copy  for  post¬ 
delivery  use.  Minimize  the  TDP  cost  by  request¬ 
ing  the  contractor’s  format  (for  example,  accept¬ 
ing  the  same  drawings  they  use  for  production), 
and  asking  only  for  details  on  items  developed  with 
government  funds. 

Data  Call  for  Government  Contracts 

As  part  of  the  development  of  an  Invitation  for 
Bid  or  Request  for  Proposals,  the  program  office 
issues  a  letter  that  describes  the  planned  procure¬ 
ment  and  asks  integrated  team  leaders  and  effected 
functional  managers  to  identify  and  justify  their 
data  requirements  for  that  contract.  A  description 
of  each  data  item  needed  is  then  developed  by  the 
affected  teams  or  functional  offices,  and  reviewed 
by  the  program  office.  Data  Item  Descriptions, 
located  in  the  Acquisition  Management  systems 
Data  List  (AMSDL)  (see  chapter  8)  can  be  used 
for  guidance  in  developing  these  descriptions. 

Concurrent  with  the  DoD  policy  on  specifica¬ 
tions  and  standards,  there  is  a  trend  to  avoid  use 


of  standard  Data  Item  Descriptions  on  contracts, 
and  specify  the  data  item  with  a  unique  tailored 
data  description  referenced  in  the  Contract  Data 
Requirements  List. 

10.5  SUMMARY  POINTS 

•  Configuration  management  is  essential  to  con¬ 
trol  the  system  design  throughout  the  life  cycle. 

•  Use  of  integrated  teams  in  an  IPPD  environ¬ 
ment  is  necessary  for  disciplined  configuration 
management  of  complex  systems. 

•  Technical  data  management  is  essential  to  trace 
decisions  and  changes  and  to  document  designs, 
processes  and  procedures. 

•  Interface  management  is  essential  to  ensure  that 
system  elements  are  compatible  in  terms  of 
form,  fit  and  function. 

•  Three  configuration  baselines  are  managed: 

-  Functional  (System  level) 

-  Allocated  (Design  To) 

-  Product  (Build  To/As  Built) 

Configuration  management  is  a  shared  responsi¬ 
bility  between  the  government  and  the  contrac¬ 
tor.  Contract  manager  (CM)  key  elements  are 
Identification,  Control,  Status  Accounting,  and 
Audits. 
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1.1  PROGRESS  MEASUREMENT 

The  Systems  Engineer  measures  design  progress 
and  maturity  by  assessing  its  development  at  key 
event-  driven  points  in  the  development  sched¬ 
ule.  The  design  is  compared  to  pre-established 
exit  criteria  for  the  particular  event  to  determine 
if  the  appropriate  level  of  maturity  has  been 
achieved.  These  key  events  are  generally  known 
as  Technical  Reviews  and  Audits. 

A  system  in  development  proceeds  through  a 
sequence  of  stages  as  it  proceeds  from  concept  to 
finished  product.  These  are  referred  to  as  “levels 
of  development.”  Technical  Reviews  are  done  after 
each  level  of  development  to  check  design  matu¬ 
rity,  review  technical  risk,  and  determines  whether 
to  proceed  to  the  next  level  of  development.  Tech¬ 
nical  Reviews  reduce  program  risk  and  ease  the 
transition  to  production  by: 

•  Assessing  the  maturity  of  the  design/ 
development  effort, 

•  Clarifying  design  requirements, 

•  Challenging  the  design  and  related  processes, 

•  Checking  proposed  design  configuration 
against  technical  requirements,  customer  needs, 
and  system  requirements, 

•  Evaluating  the  system  configuration  at  different 
stages, 

•  Providing  a  forum  for  communication,  coor¬ 
dination,  and  integration  across  all  disciplines 
and  IPTs, 


•  Establishing  a  common  configuration  baseline 
from  which  to  proceed  to  the  next  level  of 
design,  and 

•  Recording  design  decision  rationale  in  the 
decision  database. 

Formal  technical  reviews  are  proceeded  by  a  series 
of  technical  interchange  meetings  where  issues, 
problems  and  concerns  are  surfaced  and  addressed. 
The  formal  technical  review  is  not  the  place  for 
problem  solving,  but  for  verifying  that  problem 
solving  has  been  done;  it  is  a  process  rather  than 
an  event! 

Technical  Reviews  Planning 

Planning  for  Technical  Reviews  must  be  extensive 
and  up-front-and-early.  Important  considerations 
for  planning  include  the  following: 

•  Timely  and  effective  attention  and  visibility  into 
the  activities  preparing  for  the  review, 

•  Identification  and  allocation  of  resources 
necessary  to  accomplish  the  total  review  effort, 

•  Tailoring  consistent  with  program  risk  levels, 

•  Scheduling  consistent  with  availability  of 
appropriate  data, 

•  Establishing  event-driven  entry  and  exit  criteria, 

»  Where  appropriate,  conduct  of  incremental 
reviews, 

•  Implementation  by  Integrated  Product  Teams, 
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Figure  11-1.  Technical  Review  Process 
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Do  the  work  ahead  of  the  review  event.  Use  the 
review  event  as  a  confirmation  of  completed  ef¬ 
fort.  The  data  necessary  to  determine  if  the  exit 
criteria  are  satisfied  should  be  distributed,  ana¬ 
lyzed,  and  analysis  coordinated  prior  to  the  review. 
The  type  of  information  needed  for  a  technical 
review  would  include:  specifications,  drawings, 
manuals,  schedules,  design  and  test  data,  trade 
studies,  risk  analysis,  effectiveness  analyses,  mock 
ups,  breadboards,  in-process  and  finished  hard¬ 
ware,  test  methods,  technical  plans  (Manufactur¬ 
ing,  Test,  Support,  Training),  and  trend  (metrics) 
data.  Reviews  should  be  brief  and  follow  a  pre¬ 
pared  agenda  based  on  the  pre-review  analysis. 


Only  designated  participants  should  personally 
attend.  These  individuals  should  be  those  that  were 
involved  in  the  preparatory  work  for  the  review 
and  members  of  the  IPTs  responsible  for  meeting 
the  event  exit  criteria.  Participants  should  include 
representation  from  all  appropriate  government 
activities,  contractor,  subcontractors,  vendors  and 
suppliers. 

A  review  is  the  confirmation  of  a  process.  New 
items  should  not  come  up  at  the  review.  If  signifi¬ 
cant  items  do  emerge,  it’s  a  clear  sign  the  review 
is  being  held  prematurely,  and  project  risk  has 
just  increased  significantly.  A  poorly  orchestrated 
and  performed  technical  review  is  a  significant 
indicator  of  management  problems. 


Previous  DoD 

(Old  MIL-STD-1521B) 


System  Req’t  Review  (SRR) 


System  Design  Review  (SDR) 


Software  Spec  Review  (SSR) 


Preliminary  Design  Review 
(PDR) 


Critical  Design  Review  (CDR) 


EIA  IS-632 


Alternative  Systems  Review 
(ASR) 


System  Req’t  Review  (SRR) 


IEEE  PI  220 


Alternative  Concept  Review 
(ACR) 


System  Functional  Review  (SFR)  System  Definition  Review  (SDR) 


Preliminary  Design  Review 
(PDR) 


Critical  Design  Review  (CDR) 


Subsystem,  System  PDR 


Component,  Subsystem,  System 
Detail  Design  Review  (DDR) 


Test  Readiness  Review  (TRR) 


Component,  Subsystem,  System 
TRR 


Production  Readiness  Reviews 
(PRR) 


Formal  Qualification  Review 
(FOR) 


Functional  Configuration  Audit 
(FCA)  -  Replaced  by  MIL-STD-973 


Physical  Configuration  Review 
(PC A)  -  Replaced  by  MIL-STD-973 


Functional  Configuration 
Audit  (FCA) 


System  Verification  Review 
(SVR)  -  Replaced  FQR  &  PRR 


System  Physical  Configuraiton 
Review  (PCA) 


Component,  Subsystem,  System 
Production  Approval  Reviews 
(PAR) 


Component,  Subsystem,  System 
FCA 


Component,  Subsystem,  System 
PCA 
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Action  items  resulting  from  the  review  are  docu¬ 
mented  and  tracked.  These  items,  identified  by 
specific  nomenclature  and  due  dates,  are  prepared 
and  distributed  as  soon  as  possible  after  the  review. 
The  action  taken  is  tracked  and  results  distributed 
as  items  are  completed. 


Phasing  of  Technical  Reviews 


11.2  TECHNICAL  REVIEWS 


Technical  Reviews  and  Audits  are  placed  at 
strategic  event-driven  points  throughout  the  devel¬ 
opment  process,  and  passage  is  governed  by  exit 
criteria.  In  general  they  represent  a  point  where 
there  is  a  transition  in  design  focus  or  phase.  Figure 
11-3  shows  the  major  reviews  and  audits  and  how 
they  fit  into  the  development  process. 


Technical  reviews  have  various  names,  the  most 
common  of  which  are  those  identified  in  the  major 
Systems  Engineering  related  standards,  as  shown 
on  table  11-2.  The  names  used  in  reference  to 
reviews  is  unimportant;  however,  it  is  important 
that  reviews  be  held  at  appropriate  points  in  pro¬ 
gram  development  and  that  both  the  contractor  and 
government  have  common  expectations  regarding 
the  content  and  outcomes. 


Alternative  Systems/Concept  Review 
(ASR/ACR) 


After  the  concept  studies  are  complete  a  preferred 
system  concept  is  identified.  The  associated  draft 
System  Work  Breakdown  Structure,  preliminary 
functional  baseline,  and  draft  system  specification 
is  reviewed  to  determine  feasibility  and  risk.  This 
review  is  conducted  late  during  the  Concept 


Specifications 


-T--' 


System  Specification  (formerly  the  A  spec) 


Program-Unique 

Specifications 


Item  Performance  Spec  (formerly  the  B  Spec) 


A. 


Item  Detail  (C),  Process  (D) 
and  Material  (E)  Specification 


Functional  (System)  Baseline 


Configuration 

Baselines 


Major  Technical 
Reviews  &  Audits , 


1 


Allocated  (Design-To)  Baseline 


Product  (Build-To)  Baseline 


A 

A  A  A  AAA 

A 

Traditional  (MS-1 521 B) 

SRR 

A 

SSR  PDR  CDR  TRR  FCA 

SDR 

PCA 

EIA/IS-632  Reviews 

ASR 

SRR 

SFR  PDR  CDR  TRR  SVR/ 

FCAs 

PCA 

IEEE  PI 220  Reviews 

ACR 

SDefR  PDR  DDR  PARs  FCAs 

PC  As 
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Exploration  phase  to  show  that  the  preferred 
system  concept: 

•  Provides  a  cost  effective,  operationally  effective 
and  suitable  solution  to  identified  needs, 

•  Meets  established  affordability  criteria,  and 

•  Can  be  developed  to  provide  a  timely  solution 
to  the  need  at  an  acceptable  level  of  risk. 

The  findings  of  this  review  are  a  significant  input 
to  the  information  presented  at  Milestone  I. 

System  Requirements  Review  (SRR) 

At  the  beginning  of  the  Program  Definition  and 
Risk  Reduction  phase  (PDRR),  the  process  begins 
to  define  the  system  and  ensure  that  the  technol¬ 
ogy  is  available  to  design  and  produce  it.  The  initial 
requirements  analysis  will  drive  the  rest  of  the 
development.  Therefore,  it  is  appropriate  to  review 
this  analysis  to  determine  if  it  is  complete,  com¬ 
prehensive,  unambiguous,  and  free  of  conflict.  The 
review  will  consolidate  the  technical  position  of 
what  is  necessary  to  establish  a  workable  and 
achievable  system  description  based  on  review  and 
interpretation  of  requirements,  available  resources, 
and  available  technology. 

The  SRR  is  performed  either  in  late  Concept 
Exploration  or  early  PDRR  phase.  The  objective 
is  to  review  and  evaluate  the  draft  functional 
baseline  and  requirements  analysis.  All  relevant 
documentation  should  be  reviewed,  including: 

•  Feasibility  Analysis  (results  of  technology 
assessments  and  trade  studies  to  justify  system 
design  approach), 

•  System  Operational  Requirements, 

•  System  Maintenance  Concept, 

•  Functional  Analysis  (top  level  block  diagrams), 

•  Significant  system  design  criteria  (reliability, 
maintainability,  logistics  requirements,  etc.), 


•  Draft  System  Specification  and  any  initial  draft 
Performance  Item  Specifications, 

•  System  Engineering  Planning, 

•  Test  and  Evaluation  Master  Plan, 

•  Draft  top-level  Technical  Performance 
Measurement,  and 

•  System  design  documentation  (layout  draw¬ 
ings,  conceptual  design  drawings,  selected 
supplier  components  data,  etc.). 

System  Design/Definition/Functional 
Review  (SDR/SFR) 

In  PDRR,  once  most  of  the  effort  involved  in 
system  definition  is  complete,  and  the  System 
Specification  is  ready  to  be  put  under  formal 
control,  then  it’s  time  to  review  Systems  Engi¬ 
neering  Process  outputs  relating  to  the  func¬ 
tional  and  allocated  baselines.  Most  importantly, 
the  system  technical  description  (Functional 
Baseline)  must  be  approved  as  the  governing 
technical  requirement  before  proceeding  to 
further  technical  development.  The  System 
Specification  must  be  completed  to  support  the 
decision  to  begin  engineering  development 
(. Milestone  II). 

As  a  result  of  this  review,  the  System  Specifica¬ 
tion  will  be  confirmed  to  describe  the  system 
requirements,  and  the  government  will  normally 
assume  control  of  the  Functional  Baseline.  At  a 
minimum,  the  review  should  include  assessment 
of  the  following  items: 

•  Functional  Analysis  and  Allocation  of  require¬ 
ments  to  items  below  system  level, 

•  System  Specification, 

•  Draft  Item  Performance  and  some  Item  Detail 
Specifications, 

•  Design  data  defining  the  overall  system, 
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•  Verification  that  the  risks  associated  with  the 
system  design  are  at  acceptable  levels  for 
engineering  development, 

•  Verification  that  the  design  selections  have  been 
optimized  through  appropriate  trade  study 
analyses, 

•  Analyses,  reports,  “-ility”  predictions,  logistics 
support  analysis  data,  and  design  documenta¬ 
tion, 

•  Technical  Performance  Measurement  data  and 
analysis,  and 

•  Associated  enabling  product  plans  (“-ility” 
planning,  management  plans,  human  factors 
management  plans,  etc.). 

Software  Specification  Review  (SSR) 

To  prepare  for  PDR,  software  issues  are  examined 
and  consolidated  prior  to  establishing  the  Allocated 
Baseline.  This  review  is  performed  early  in  the 
Engineering  and  Manufacturing  Development 
phase,  prior  to  a  system-level  PDR.  The  objective 
is  to: 

•  Review  and  evaluate  the  maturity  of  software 
requirements, 

•  Validate  software-related  Item  Performance 
Specifications, 

•  Establish  software  specific  requirements  to  be 
included  in  allocated  baseline, 

•  Examine  Software  Requirements  and  Interface 
Requirements  Specifications,  and 

•  Examine  the  Operations  Concept  Document. 

Preliminary  Design  Review  (PDR) 

Using  the  Functional  Baseline,  especially  the  Sys¬ 
tem  Specification,  as  a  governing  requirement,  a 
preliminary  design  is  expressed  in  terms  of  design 
requirements  for  subsystems  and  configuration 
items.  This  preliminary  design  sets  forth  the 


functions,  performance,  and  interface  require¬ 
ments  that  will  govern  design  of  the  items  be¬ 
low  system  level.  Following  the  PDR,  this  pre¬ 
liminary  design  (Allocated  Baseline)  will  be  put 
under  formal  configuration  control  [usually]  by 
the  contractor.  The  Item  Performance  Speci¬ 
fications,  which  form  the  core  of  the  Allo¬ 
cated  Baseline,  will  be  confirmed  to  represent 
a  design  that  meets  the  System  Specification. 

This  review  is  performed  early  in  the  Engineering 
and  Manufacturing  Development  phase.  Reviews 
are  held  for  configuration  items  (CIs),  or  groups 
of  related  CIs,  prior  to  a  system-level  PDR.  Item 
Performance  Specifications  are  put  under  configu¬ 
ration  control.  (Latest  DoD  practice  is  for  contrac¬ 
tors  to  maintain  configuration  control  over  Item 
Performance  Specifications,  while  the  govern¬ 
ment  exercises  requirements  control  at  the  system 
level.)  At  a  minimum,  the  review  should  include 
assessment  of  the  following  items: 

•  Item  Performance  Specifications, 

•  Draft  Item  Detail,  Process,  and  Material 
Specifications, 

•  Design  data-defining  major  subsystems, 
equipment,  software,  and  other  system 
elements, 

•  Analyses,  reports,  “-ility”  analyses,  trade  stud¬ 
ies,  logistics  support  analysis  data,  and  design 
documentation, 

•  Technical  Performance  Measurement  data  and 
analysis, 

•  Engineering  breadboards,  laboratory  models, 
test  models,  mockups,  and  prototypes  used  to 
support  the  design,  and 

•  Supplier  data  describing  specific  components. 

[Rough  Rule  of  Thumb:  -15%  of  production 
drawings  are  released  by  PDR.  This  rule  is 
anecdotal  and  only  guidance  relating  to  an 
“ average ”  defense  hardware  program.] 
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Critical/Detail  Design  Review  (CDR/DDR) 

Before  starting  to  build  the  production  line  there 
needs  to  be  verification  and  formalization  of  the 
mutual  understanding  of  the  details  of  the  item 
being  produced.  Performed  during  the  Engineer¬ 
ing  and  Manufacturing  Development  phase,  this 
review  evaluates  the  draft  Production  Baseline 
(“Build  To”  documentation)  to  determine  if  the 
system  design  documentation  (Product  Baseline, 
Item  Detail  Specs,  Material  Specs,  Process  Specs) 
is  satisfactory  to  start  initial  manufacturing.  This 
review  includes  the  evaluation  of  all  configuration 
items.  It  includes  a  series  of  reviews  conducted 
for  each  hardware  Cl  before  release  of  design  to 
fabrication,  and  each  Computer  Software  Cl  be¬ 
fore  final  coding  and  testing.  Additionally,  test 
plans  are  reviewed  to  assess  if  test  efforts  are  de¬ 
veloping  sufficiently  to  indicate  the  Test  Readi¬ 
ness  Review  will  be  successful.  The  approved  de¬ 
tail  design  serves  as  the  basis  for  final  production 
planning  and  initiates  the  development  of  final 
software  code. 

[Rough  Rule  of  Thumb:  At  CDR  the  design 
should  be  ~  85%  complete.  Many  programs  use 
drawing  release  as  a  metric  for  measuring  design 
completion.  This  rule  is  anecdotal  and  only  guid¬ 
ance  relating  to  an  “average”  defense  hardware 
program.] 

Test  Readiness  Review  (TRR) 

Performed  late  in  the  Engineering  and  Manufac¬ 
turing  Development  phase  (after  CDR),  the  Test 
Readiness  Review  assesses  test  objectives,  proce¬ 
dures,  and  resources  testing  coordination.  Origi¬ 
nally  developed  as  a  software  Cl  review,  this  review 
is  increasingly  applied  to  both  hardware  and  soft¬ 
ware  items.  The  TRR  determines  the  complete¬ 
ness  of  test  procedures  and  their  compliance  with 
test  plans  and  descriptions.  Completion  coincides 
with  the  initiation  of  formal  Cl  testing. 

Production  Readiness/Approval 
Reviews  (PRR/PAR) 

Performed  incrementally  during  the  Engineering 
and  Manufacturing  Development  phase,  this  series 


of  reviews  is  held  to  determine  if  production 
preparation  for  the  system,  subsystems,  and  con¬ 
figuration  items  is  complete,  comprehensive,  and 
coordinated.  PRRs  are  necessary  to  determine  the 
readiness  for  production  prior  to  executing  a 
production  go-ahead  decision.  They  will  formally 
examine  the  producibility  of  the  production  design, 
the  control  over  the  projected  production  processes, 
and  adequacy  of  resources  necessary  to  execute 
production.  Manufacturing  risk  is  evaluated  in 
relationship  to  product  and  manufacturing  process 
performance,  cost,  and  schedule.  These  reviews 
support  acquisition  decisions  to  proceed  to  Low- 
Rate  Initial  Production  or  Full-Rate  Production. 

Functional  Configuration  Audit/  System 
Verification  Review  (FCA)/(SVR) 

This  series  of  audits  and  the  consolidating  Sys¬ 
tem  Verification  Review  re-examines  and  verifies 
the  customer’s  needs,  and  the  relationship  of  these 
needs  to  the  system  and  subsystem  technical  per¬ 
formance  descriptions  (Functional  and  Allocated 
Baselines).  They  determine  if  the  system  produced 
(including  production  representative  prototypes  or 
LRIP  units)  is  capable  of  meeting  the  technical 
performance  requirements  established  in  the 
specifications,  test  plans,  etc.  The  technical  assess¬ 
ments  and  decisions  that  are  made  in  SVR  will  be 
presented  to  support  the  Milestone  III  full  rate 
production  go-ahead  decision.  Among  the  issues 
addressed: 

•  Readiness  issues  for  continuing  design,  con¬ 
tinuing  verifications,  production,  training, 
deployment,  operations,  support,  and  disposal 
have  been  resolved, 

•  Verification  is  comprehensive  and  complete, 

•  Configuration  audits,  including  completion  of 
all  change  actions,  have  been  completed  for  all 
CIs, 

•  Risk  management  planning  is/has  been 
updated  for  production, 

•  Systems  Engineering  planning  is  updated  for 
production,  and 
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•  Critical  achievements,  success  criteria  and 
metrics  have  been  established  for  production. 

Physical  Configuration  Audit  (PCA) 

After  full  rate  production  has  been  approved, 
follow-on  independent  verification  (FOT&E)  has 
identified  the  changes  the  user  requires,  and  those 
changes  have  been  corrected  on  the  baseline  docu¬ 
ments  and  the  production  line,  then  it  is  time  to 
ensure  that  the  product  and  the  product  baseline 
correspond.  This  audit  will  formalize  the  Product 
Baseline,  including  specifications  and  the  techni¬ 
cal  data  package,  so  that  future  changes  can  only 
be  made  through  full  configuration  management 
procedures.  Fundamentally,  the  PCA  verifies  the 
product  (as  built)  is  consistent  with  all  baseline 
documentation,  including  the  Technical  Data  Pack¬ 
age  which  describes  the  Product  Baseline.  The  final 
PCA  confirms: 

•  The  subsystem  and  Cl  PCAs  have  been 
successfully  completed, 

•  The  integrated  decision  data  base  is  valid  and 
represents  the  product, 

•  All  items  have  been  baselined, 

•  Changes  to  previous  baselines  have  been 
completed, 

•  Testing  deficiencies  have  been  resolved  and 
appropriate  changes  implemented,  and 

•  System  processes  are  current  and  can  be 
executed. 

The  PCA  is  a  configuration  management  activity 
and  is  conducted  following  procedures  established 
in  the  Configuration  Management  Plan. 

11.3  TAILORING 

The  reviews  laid  out  above  are  based  on  a  com¬ 
plex  system  development  project  requiring  signifi¬ 
cant  technical  evolution.  There  are  also  cases  where 
system  technical  maturity  is  more  advanced  than 


normal  for  the  phase,  for  example,  where  a  pre¬ 
vious  cancelled  program  or  an  Advanced  Tech¬ 
nical  Concept  Demonstration  (ACTD)  has  pro¬ 
vided  a  significant  level  of  technical  develop¬ 
ment  applicable  to  the  current  program.  In  some 
cases  this  will  precipitate  the  merging  or  even 
elimination  of  acquisition  phases.  This  does  not 
justify  elimination  of  the  technical  management 
activities  grouped  under  the  general  heading  of 
systems  analysis  and  control,  nor  does  it  relieve 
the  government  program  manager  of  the  respon¬ 
sibility  to  see  that  these  disciplines  are  enforced. 
It  does,  however,  highlight  the  need  for  flexibil¬ 
ity  and  tailoring  to  the  specific  needs  of  the  pro¬ 
gram  under  development. 

For  example,  a  DoD  acquisition  strategy  that 
proposes  a  combined  Milestone  I/II  may  skip  a 
milestone,  but  it  must  not  skip  the  formulation  of 
an  appropriate  Functional  Baseline  and  the  equiva¬ 
lent  of  an  SFR  to  support  the  combined  milestone. 
Nor  should  it  skip  the  formulation  of  the  Allo¬ 
cated  Baseline  and  the  equivalent  of  a  PDR,  and 
the  formulation  of  the  Product  Baseline  and  the 
equivalent  of  a  CDR  after  the  milestone  decision. 

Baselines  must  be  developed  sequentially  because 
they  document  different  levels  or  types  of  detail 
and  must  build  on  each  other.  However,  the 
assessment  of  design  and  development  maturity 
can  be  tailored  as  appropriate  for  the  particular 
system.  Tailored  efforts  still  have  to  deal  with 
the  problem  of  determining  when  the  design 
maturity  should  be  assessed,  and  how  these 
assessments  will  support  the  formulation  and 
control  of  baselines. 

In  tailoring  efforts,  be  extremely  careful  determin¬ 
ing  the  level  of  system  complexity.  The  system 
integration  effort,  the  development  of  a  single, 
advanced  technology  or  complex  sub-component, 
or  the  need  for  intensive  software  development  may 
be  sufficient  to  establish  the  total  system  as  a  com¬ 
plex  project,  even  though  it  appears  simple  because 
most  subsystems  are  simple  or  off-the-shelf. 


96 


Chapter  11 


Technical  Reviews  and  Audits 


11.4  SUMMARY  POINTS 

•  Each  level  of  product  development  is  evaluated 
and  progress  is  controlled  by  specification 
development  (System,  Item  Performance,  Item 
Detail,  Process,  and  Material  specifications)  and 
technical  reviews  and  audits  (ASR,  SRR,  SDR, 
SSR,  PDR,  CDR,  TRR,  PRR,  FCA,  SVR, 
PCA). 

•  Technical  reviews  assess  development  matur¬ 
ity,  risk,  and  cost/schedule  effectiveness  to 
determine  readiness  to  proceed. 

•  Reviews  must  be  planned,  managed,  and  fol¬ 
lowed  up  to  be  effective  as  an  analysis-and- 
control  tool. 


•  As  the  system  progresses  through  the  devel¬ 
opment  effort,  the  nature  of  design  reviews 
and  audits  will  parallel  the  technical  effort. 
Initially  they  will  focus  on  requirements  and 
functions,  and  later  become  very  product  fo¬ 
cused. 

•  After  system  level  reviews  establish  the  Func¬ 
tional  Baseline,  technical  reviews  tend  to  be 
subsystem  and  Cl  focused  until  late  in  devel¬ 
opment  when  the  focus  again  turns  to  the  sys¬ 
tem  level  to  determine  the  system’s  readiness 
for  production. 
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12.1  MAKING  CHOICES 

Trade  Studies  are  a  formal  decision  making 
methodology  used  by  integrated  teams  to  make 
choices  and  resolve  conflicts  during  the  systems 
engineering  process.  Good  trade  study  analyses 
demand  the  participation  of  the  integrated  team; 
otherwise,  the  solution  reached  may  be  based 
on  unwarranted  assumptions  or  may  reflect  the 
omission  of  important  data. 

Trade  studies  identify  desirable  and  practical 
alternatives  among  requirements,  technical  ob¬ 
jectives,  design,  program  schedule,  functional 
and  performance  requirements,  and  life  cycle 
costs  are  identified  and  conducted.  Choices  are 
then  made  using  a  defined  set  of  criteria.  Trade 
studies  are  defined,  conducted,  and  documented 
at  the  various  levels  of  the  functional  or  physi¬ 
cal  architecture  in  enough  detail  to  support  deci¬ 
sion  making  and  lead  to  a  balanced  system  solu¬ 
tion.  The  level  of  detail  of  any  trade  study  needs 
to  be  commensurate  with  cost,  schedule,  perfor¬ 
mance,  and  risk  impacts. 

Both  formal  and  informal  trade  studies  are 
conducted  in  any  systems  engineering  activity. 
Formal  trade  studies  tend  to  be  those  that  will 
be  used  in  formal  decision  forums,  e.g.,  mile¬ 
stone  decisions.  These  are  typically  well-docu¬ 
mented  and  become  a  part  of  the  decision  data 
base  normal  to  systems  development.  On  the  other 
hand,  engineering  choices  at  every  level  involve 
trade-offs  and  decisions  that  parallel  the  trade 
study  process.  Most  of  these  less  formal  studies 
are  documented  in  summary  detail  only,  but  they 
are  important  in  that  they  define  the  design  as  it 
evolves. 


Systems  Engineering  Process 
and  Trade  Studies 

Trade  studies  are  required  to  support  decisions 
throughout  the  systems  engineering  process. 
During  requirements  analysis,  requirements  are 
balanced  against  other  requirements  or  con¬ 
straints,  including  cost.  Requirements  analysis 
trade  studies  examine  and  analyze  alternative 
performance  and  functional  requirements  to  re¬ 
solve  conflicts  and  satisfy  customer  needs. 

During  functional  analysis  and  allocation,  func¬ 
tions  are  balanced  with  interface  requirements, 
dictated  equipment,  functional  partitioning, 
requirements  flowdown,  and  configuration  items 
designation  considerations.  Trade  studies  are 
conducted  within  and  across  functions  to: 

•  Support  functional  analyses  and  allocation 
of  performance  requirements  and  design 
constraints, 

•  Define  a  preferred  set  of  performance  require¬ 
ments  satisfying  identified  functional  inter¬ 
faces, 

•  Determine  performance  requirements  for 
lower-level  functions  when  higher-level  per¬ 
formance  and  functional  requirements  can¬ 
not  be  readily  resolved  to  the  lower-level,  and 

•  Evaluate  alternative  functional  architectures. 

During  design  synthesis,  trade  studies  are  used 
to  evaluate  alternative  solutions  to  optimize  cost, 
schedule,  performance,  and  risk.  Trade  studies 
are  conducted  during  synthesis  to: 
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•  Establish  system,  subsystem,  and  component 
configurations; 

•  Assist  in  selecting  system  concepts,  designs, 
and  solutions  (including  people,  parts,  and 
materials  availability); 

•  Support  materials  selection  and  make-or-buy, 
process,  rate,  and  location  decisions; 

•  Examine  proposed  changes; 

•  Examine  alternative  technologies  to  satisfy 
functional  or  design  requirements  including 
alternatives  for  moderate-  to  high-risk 
technologies; 

•  Evaluate  environmental  and  cost  impacts  of 
materials  and  processes; 

•  Evaluate  alternative  physical  architectures  to 
select  preferred  products  and  processes;  and 

•  Select  standard  components,  techniques, 
services,  and  facilities  that  reduce  system  life 
cycle  cost  and  meet  system  effectiveness 
requirements. 

During  Concept  Exploration  and  functional  base¬ 
line  development  (CE  and  PDRR)  trade  studies 
are  used  to  examine  alternative  system  level  con¬ 
cepts  and  scenarios  to  help  establish  the  system 
configuration.  During  allocated  and  product  base¬ 
line  developments  (EMD)  trade  studies  examine 
lower-level  system  segments,  subsystems,  and 
end  items  to  assist  in  selecting  component  part 
designs.  Performance,  cost,  safety,  reliability, 
risk,  and  other  effectiveness  measures  must 
be  traded  against  each  other  and  against  physi¬ 
cal  characteristics. 


12.2  TRADE  STUDY  BASICS 

Trade  studies  (trade-off  analyses)  are  processes 
that  examine  viable  alternatives  to  determine 
which  is  preferred.  It  is  important  that  there  be 
criteria  established  that  are  acceptable  to  all 
members  of  the  integrated  team  as  a  basis  for  a 


decision.  In  addition,  there  must  be  an  agreed 
upon  approach  to  measuring  alternatives  against 
the  criteria.  If  these  principles  are  followed,  the 
trade  study  should  produce  decisions  that  are 
rational,  objective,  and  repeatable.  Finally,  trade 
study  results  must  be  such  that  they  can  be  eas¬ 
ily  communicated  to  customers  and  decision 
makers.  If  the  results  of  a  trade  study  are  too 
complex  to  communicate  with  ease,  it  is  unlikely 
that  the  process  will  result  in  timely  decisions. 

Trade  Study  Process 

As  shown  by  Figure  12-1,  the  process  of  trade¬ 
off  analysis  consists  of  defining  the  problem, 
bounding  the  problem,  establishing  a  trade-off 
methodology  (to  include  the  establishment  of 
decision  criteria),  selecting  alternative  solutions, 
determining  the  key  characteristics  of  each  al¬ 
ternative,  evaluating  the  alternatives,  and  choos¬ 
ing  a  solution: 

•  Defining  the  problem  entails  developing  a 
problem  statement  including  any  constraints. 
Problem  definition  should  be  done  with  ex¬ 
treme  care.  After  all,  if  you  don’t  have  the 
right  problem,  you  won’t  get  the  right  an¬ 
swer. 

•  Bounding  and  understanding  the  problem 
requires  identification  of  system  requirements 
that  apply  to  the  study  conflicts  between 
desired  characteristics  of  the  product  or  pro¬ 
cess  being  studied,  and  the  limitations  of  avail¬ 
able  data.  Available  databases  should  be  iden¬ 
tified  that  can  provide  relevant,  historical  “ac¬ 
tual”  information  to  support  evaluation  deci¬ 
sions. 

•  Establishing  the  methodology  includes 
choosing  the  mathematical  method  of  com¬ 
parison,  developing  and  quantifying  the  cri¬ 
teria  used  for  comparison,  and  determining 
weighting  factors  (if  any).  Use  of  appropri¬ 
ate  models  and  methodology  will  dictate  the 
rationality,  objectivity,  and  repeatability  of  the 
study.  Experience  has  shown  that  this  step 
can  be  easily  abused  through  both  ignorance 
and  design.  To  the  extent  possible  the  chosen 
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Figure  12-1.  Trade  Study  Process 


methodology  should  compare  alternatives 
based  on  true  value  to  the  customer  and  de¬ 
veloper.  Trade-off  relationships  should  be  rel¬ 
evant  and  rational.  Choice  of  utility  or 
weights  should  answer  the  question,  “What  is 
the  actual  value  of  the  increased  performance, 
based  on  what  rationale?” 

•  Selecting  alternative  solutions  requires  iden¬ 
tification  of  all  the  potential  ways  of  solving 
the  problem  and  selecting  those  that  appear 
viable.  The  number  of  alternatives  can  drive 
the  cost  of  analysis,  so  alternatives  should 
normally  be  limited  to  clearly  viable  choices. 

•  Determining  the  key  characteristics  entails 
deriving  the  data  required  by  the  study 
methodology  for  each  alternative. 


•  Evaluating  the  alternatives  is  the  analysis  part 
of  the  study.  It  includes  the  development  of  a 
trade-off  matrix  to  compare  the  alternatives, 
performance  of  a  sensitivity  analysis,  selec¬ 
tion  of  a  preferred  alternative,  and  a  re-evalu¬ 
ation  (sanity  check)  of  the  alternatives  and 
the  study  process.  Since  weighting  factors  and 
some  “quantified”  data  can  have  arbitrary  as¬ 
pects,  the  sensitivity  analysis  is  crucial.  If  the 
solution  can  be  changed  with  relatively  mi¬ 
nor  changes  in  data  input,  the  study  is  prob¬ 
ably  invalid,  and  the  methodology  should  be 
reviewed  and  revised.  After  the  above  tasks 
are  complete,  a  solution  is  chosen,  docu¬ 
mented,  and  recorded  in  the  database. 
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performance  to  its  cost.  These  analyses  help 
determine  affordability  and  relative  values  of 
alternate  solutions.  Specifically,  they  are  used 
to: 

•  Support  identification  of  affordable,  cost  op¬ 
timized  mission  and  performance  require¬ 
ments. 

•  Support  the  allocation  of  performance  to  an 
optimum  functional  structure. 

•  Provide  criteria  for  the  selection  of  alterna¬ 
tive  solutions. 


12.3  SUMMARY  POINTS 

•  The  purpose  of  trade  studies  is  to  make  bet¬ 
ter  and  more  informed  decisions  in  selecting 
best  alternative  solutions. 

•  Initial  trade  studies  focus  on  alternative  sys¬ 
tem  concepts  and  requirements.  Later  studies 
assist  in  selecting  component  part  designs. 

•  Cost  effectiveness  analyses  provide  assess¬ 
ments  of  alternative  solution  performance 
relative  to  cost. 


•  Provide  analytic  confirmation  that  designs 
satisfy  customer  requirements  within  cost 
constraints. 

•  Support  product  and  process  verification. 
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SUPPLEMENT  A 

UTILITY  CURVE 
METHODOLOGY 


The  utility  curve  is  a  common  methodology  used 
in  DoD  and  industry  to  perform  trade-off  analy¬ 
sis.  In  DoD  it  is  widely  used  for  cost  effective¬ 
ness  analysis  and  proposal  evaluation. 

Utility  Curve 

The  method  uses  a  utility  curve,  Figure  12-2, 
for  each  of  the  decision  factors  to  normalize  them 
to  ease  comparison.  This  method  establishes  the 
relative  value  of  the  factor  as  it  increases  from 
the  minimum  value  of  the  range.  Depending  on 
the  decision  factor  employed,  the  curve  shows  a 
constant  value  relationship  (straight  line),  in¬ 
creasing  value  (concave  curve),  decreasing  value 
(convex  curve),  or  a  stepped  value. 

Decision  Matrix 


are  used  to  establish  weighting  factors  for  each 
decision  factor.  The  weighting  factors  prioritize 
the  decision  factors  and  allow  direct  compari¬ 
son  between  them.  A  decision  matrix,  similar  to 
Figure  12-3,  is  generated  to  evaluate  the  relative 
value  of  the  alternative  solutions.  In  the  case  of 
Figure  12-3,  range  is  given  a  weight  of  2.0;  speed 
a  weight  of  1.0;  and  payload  a  weight  of  2.5. 
The  utility  values  for  each  of  the  decision  fac¬ 
tors  are  multiplied  by  the  appropriate  weight. 
The  weighted  values  for  each  alternative  solu¬ 
tion  are  added  to  obtain  a  total  score  for  each 
solution.  The  solution  with  the  highest  score  be¬ 
comes  the  preferred  solution.  For  the  transport 
analysis  of  Figure  12-3  the  apparent  preferred 
solution  is  System  3. 

Sensitivity 


Each  of  the  decision  factors  will  also  have  rela-  Figure  12-3  also  illustrates  a  problem  with  the 
tive  value  between  them.  These  relative  values  utility  curve  method.  Both  the  utility  curve  and 


Decision  Factor 
(e.g.,  speed,  cost,  reliability,  etc.) 


Figure  12-2.  Utility  Curve 
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weighting  factors  contain  a  degree  of  judgement 
that  can  vary  between  evaluators.  Figure  12-3 
shows  three  systems  clustered  around  3.8,  indi¬ 
cating  that  a  small  variation  in  the  utility  curve 
or  weighting  factor  could  change  the  results.  In 
the  case  of  Figure  12-3,  a  sensitivity  analysis 
should  be  performed  to  determine  how  solutions 
change  as  utility  and  weighting  change.  This  will 
guide  the  evaluator  in  determining  how  to  ad¬ 
just  evaluation  criteria  to  eliminate  the  problem’s 
sensitivity  to  small  changes.  In  the  case  of  Fig¬ 
ure  12-3  the  solution  could  be  as  simple  as  re¬ 
evaluating  weighting  factors  to  express  better 
the  true  value  to  the  customer.  For  example,  if 
the  value  of  range  is  considered  to  be  less  and 


payload  worth  more  than  originally  stated,  then 
System  4  may  become  a  clear  winner. 

Notes 

When  developing  or  adjusting  utility  curves  and 
weighting  factors,  communication  with  the  cus¬ 
tomers  and  decision-makers  is  essential. 
Mostsensitivity  problems  are  not  as  blatant  as 
Figure  12-3.  Sensitivity  need  not  be  apparent  in 
the  alternatives’  total  score.  To  ensure  study  vi¬ 
ability,  sensitivity  analysis  should  always  be  done 
to  examine  the  consequences  of  methodology 
choice.  (Most  decision  support  software  provides 
a  sensitivity  analysis  feature.) 


^\Decision  Factors 

Range 

Speed 

Payload 

Wt.  : 

=  2.0 

Wt.  : 

1.0 

Wt. 

=  2.5 

Weighted 

Total 

Alternatives 

U 

W 

U 

W 

U 

W 

Transport  System  1 

.8 

1.6 

.7 

.7 

.6 

1.5 

3.8 

Transport  System  2 

.7 

1.4 

.9 

.9 

.4 

1.0 

3.3 

Transport  System  3 

.6 

1.2 

.7 

.7 

.8 

2.0 

3.9 

Transport  System  4 

.5 

1.0 

.5 

.5 

.9 

2.25 

3.75 

Key:  U=  Utility  value 

W  =  Weighted  value 

Figure  12-3.  Sample  Decision  Matrix 
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MODELING  AND 
SIMULATION 


13.1  INTRODUCTION 

A  model  is  a  physical,  mathematical  or  logical 
representation  of  a  system  entity,  phenomenon,  or 
process.  A  simulation  is  the  implementation  of  a 
model  over  time.  A  simulation  brings  a  model  to 
life  and  shows  how  a  particular  object  or  phenom¬ 
enon  will  behave.  It  is  useful  for  testing,  analysis 
or  training  where  real-world  systems  or  concepts 
can  be  represented  by  a  model. 

Modeling  and  simulation  (M&S)  provides  virtual 
duplication  of  products  and  processes,  and 


represents  those  products  or  processes  in  readily 
available  and  operationally  valid  environments. 
Use  of  models  and  simulations  can  reduce  the  cost 
and  risk  of  life  cycle  activities.  As  shown  by  Figure 
13-1  the  advantages  are  significant  through  out  the 
life  cycle. 

Modeling,  Simulation,  and  Acquisition 

Modeling  and  simulation  has  become  a  very 
important  tool  across  all  acquisition  cycle  phases 
and  all  applications:  requirements  definition; 
program  management;  design  and  engineering; 


$  Savings 


Prove  System  Need: 

Use  existing  high  resolution 
modeisto  emulate 
operational  situation 


Smooth  Transition  to  Operation 

•  Manual  proven 

•  Trained  personnel 

•  Operationally  ready  before 
equipment  is  given  to 
operators 


Shortens 

Schedules 


Need 


Production 

Deployment 

O&S 


Saves  Time 


Concepts 


Test  “concepts”  in  the  “real 
world”  of  simulation  using 
simple  models  and  putting 
operators  into  process 


Reduce  Program  Risks 

•  Design 

•  Integration 

*  Transition  to  production 

*  Testing 


Detail 

Design 


& 


Prelim 

Design 


Improves  IPPD 


Helps  Refine  Requirements 

•  Get  the  user  involved 

*  Prevent  gold-plating 


Sometimes  it’s  the  only  way 
to  verify  or  validate 


Figure  13-1.  Advantages  of  Modeling  and  Simulation 
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efficient  test  planning;  result  prediction;  supple¬ 
ment  to  actual  test  and  evaluation;  manufacturing; 
and  logistics  support.  With  so  many  opportunities 
to  use  M&S,  its  four  major  benefits;  cost  savings, 
accelerated  schedule,  improved  product  quality  and 
cost  avoidance  can  be  achieved  in  any  system 
development  when  appropriately  applied.  DoD  and 
industry  around  the  world  have  recognized  these 
opportunities,  and  many  are  taking  advantage  of 
the  increasing  capabilities  of  computer  and  infor¬ 
mation  technology.  M&S  is  now  capable  of 
prototyping  full  systems,  networks,  interconnect¬ 
ing  multiple  systems  and  their  simulators  so  that 
simulation  technology  is  moving  in  every  direction 
conceivable. 


13.2  CLASSES  OF  SIMULATIONS 

The  three  classes  of  models  and  simulations  are 
virtual,  constructive,  and  live: 

•  Virtual  simulations  represent  systems  both 
physically  and  electronically.  Examples  are  air¬ 
craft  trainers,  the  Navy’s  Battle  Force  Tactical 
Trainer,  Close  Combat  Tactical  Trainer,  and 
built-in  training. 

•  Constructive  simulations  represent  a  system 
and  its  employment.  They  include  computer 
models,  analytic  tools,  mockups,  IDEF,  Flow 
Diagrams,  and  CAD/CAM. 

•  Live  simulations  are  simulated  operations  with 
real  operators  and  real  equipment.  Examples 
are  fire  drills,  operational  tests,  and  initial 
production  run  with  soft  tooling. 

Virtual  Simulation 

Virtual  simulations  put  the  human-in-the-loop. 
The  operator’s  physical  interface  with  the  system 
is  duplicated,  and  the  simulated  system  is  made  to 
perform  as  if  it  were  the  real  system.  The  operator 
is  subjected  to  an  environment  that  looks,  feels, 
and  behaves  like  the  real  thing.  The  more  advanced 
version  of  this  is  the  virtual  prototype,  which 
allows  the  individual  to  interface  with  a  virtual 
mockup  operating  in  a  realistic  computer-gener¬ 


ated  environment.  A  virtual  prototype  is  a  com¬ 
puter-based  simulation  of  a  system  or  subsystem 
with  a  degree  of  functional  realism  that  is  compa¬ 
rable  to  that  of  a  physical  prototype. 

Constructive  Simulations 

The  purpose  of  systems  engineering  is  to  develop 
descriptions  of  system  solutions.  Accordingly, 
constructive  simulations  are  important  products  in 
all  key  system  engineering  tasks  and  activities.  Of 
special  interest  to  the  systems  engineer  are  com¬ 
puter-aided  engineering  tools.  Computer-aided 
tools  can  allow  more  in-depth  and  complete  analy¬ 
sis  of  system  requirements  early  in  design.  They 
can  provide  improved  communication  because 
data  can  be  disseminated  rapidly  to  several 
individuals  concurrently,  and  because  design 
changes  can  be  incorporated  and  distributed 
expeditiously.  Key  computer-aided  engineering 
tools  are  Computer-Aided  Design,  Computer- 
Aided  Engineering,  Computer-Aided  Manufactur¬ 
ing,  Continuous  Acquisition  and  Life  Cycle  Sup¬ 
port,  and  Computer-Aided  Systems  Engineering: 

Computer-Aided  Design  (CAD).  CAD  tools  are 
used  to  describe  the  product  electronically  to 
facilitate  and  support  design  decisions.  It  can 
model  diverse  aspects  of  the  system  such  as  how 
components  can  be  laid  out  on  electrical/electronic 
circuit  boards,  how  piping  or  conduit  is  routed,  or 
how  diagnostics  will  be  performed.  It  is  used  to 
lay  out  systems  or  components  for  sizing,  posi¬ 
tioning,  and  space  allocating  using  two  or  three- 
dimensional  displays.  It  uses  three-dimensional 
“solid”  models  to  ensure  that  assemblies,  surfaces, 
intersections,  interfaces,  etc.  are  clearly  defined. 
Most  CAD  tools  automatically  generate  isometric 
and  exploded  views  of  detailed  dimensional  and 
assembly  drawings,  and  determine  component  sur¬ 
face  areas,  volumes,  weights,  moments  of  inertia, 
centers  of  gravity,  etc.  Additionally,  many  CAD 
tools  can  develop  three-dimensional  models  of 
facilities,  operator  consoles,  maintenance  work¬ 
stations,  etc.  for  evaluating  man-machine  inter¬ 
faces.  CAD  tools  are  available  in  numerous  vari¬ 
eties,  reflecting  different  degrees  of  capabilities, 
fidelity,  and  cost.  The  commercial  CAD/CAM 
product,  CATIA,  was  used  to  develop  the  Boeing 
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777,  and  is  a  good  example  of  current  state-of- 
the-art  CAD. 

Computer-Aided  Engineering  (CAE).  Computer- 
Aided  Engineering  provides  automation  of  require¬ 
ments  and  performance  analyses  in  support  of  trade 
studies.  It  normally  would  automate  technical 
analyses  such  as  stress,  thermodynamic,  acoustic, 
vibration,  or  heat  transfer  analysis.  Additionally, 
it  can  provide  automated  processes  for  functional 
analyses  such  as  fault  isolation  and  testing,  failure 
mode,  and  safety  analyses.  CAE  can  also  provide 
automation  of  life-cycle-oriented  analysis  nec¬ 
essary  to  support  the  design.  Maintainability, 
producibility,  human  factor,  logistics  support,  and 
value/cost  analyses  are  available  with  CAE  tools. 

Computer-Aided  Manufacturing  (CAM).  CAM 
tools  are  generally  designed  to  provide  automated 
support  to  both  production  process  planning  and 
to  the  project  management  process.  Process  plan¬ 
ning  attributes  of  CAM  include  establishing 
Numerical  Control  parameters,  controlling 
machine  tools  using  pre-coded  instructions,  pro¬ 
gramming  robotic  machinery,  handling  material, 
and  ordering  replacement  parts.  The  production 
management  aspect  of  CAM  provides  management 
control  over  production-relevant  data,  uses  histori¬ 
cal  actual  costs  to  predict  cost  and  plan  activities, 
identifies  schedule  slips  or  slack  on  a  daily  basis, 
and  tracks  metrics  relative  to  procurement, 
inventory,  forecasting,  scheduling,  cost  reporting, 
support,  quality,  maintenance,  capacity,  etc.  A 
common  example  of  a  computer-based  project 
planning  and  control  tool  is  Manufacturing 
Resource  Planning  II  (MRP  II).  Some  CAM  pro¬ 
grams  can  accept  data  direct  from  a  CAD  program. 
With  this  type  of  tool,  generally  referred  to  as 
CAD/CAM,  substantial  CAM  data  is  automati¬ 
cally  generated  by  importing  the  CAD  data  directly 
into  the  CAM  software. 

Computer-Aided  Systems  Engineering  (CASE). 
CASE  tools  provide  automated  support  for  the 
Systems  Engineering  and  associated  processes. 
CASE  tools  can  provide  automated  support  for 
integrating  system  engineering  activities,  perform¬ 
ing  the  systems  engineering  tasks  outlined  in 
previous  chapters,  and  performing  the  systems 
analysis  and  control  activities.  It  provides  tech¬ 


nical  management  support  and  has  a  broader 
capability  than  either  CAD  or  CAE.  An  increas¬ 
ing  variety  of  CASE  tools  are  available,  as 
competition  brings  more  products  to  market,  and 
many  of  these  support  the  commercial  best 
Systems  Engineering  practices. 

Continuous  Acquisition  and  Life  Cycle  Support 
(CALS).  CALS  relates  to  the  application  of 
computerized  technology  to  plan  and  implement 
support  functions.  The  emphasis  is  on  informa¬ 
tion  relating  to  maintenance,  supply  support,  and 
associated  functions.  An  important  aspect  of  CALS 
is  the  importation  of  information  developed  during 
design  and  production.  A  key  CALS  function  is  to 
support  the  maintenance  of  the  system  configura¬ 
tion  during  the  operation  and  support  phase.  In 
DoD,  CALS  supports  activities  of  the  logistics 
community  rather  than  the  specific  program  office, 
and  transfer  of  data  between  the  CAD  or  CAM 
programs  to  CALS  has  been  problematic.  As  a 
result  there  is  current  emphasis  on  development 
of  standards  for  compatible  data  exchange.  For¬ 
mats  of  import  include:  two-  and  three-dimen¬ 
sional  models  (CAD),  ASCII  formats  (Techni¬ 
cal  Manuals),  two-dimensional  illustrations 
(Technical  Manuals),  and  Engineering  Drawing 
formats  (Raster,  Aperture  cards).  These  formats 
will  be  employed  in  the  Integrated  Data  Envi¬ 
ronment  (IDE)  that  is  mandated  for  use  in  DoD 
program  offices. 

Live  Simulation 

Live  simulations  are  simulated  operations  of  real 
systems  using  real  people  in  realistic  situations. 
The  intent  is  to  put  the  system,  including  its 
operators,  through  an  operational  scenario,  where 
some  conditions  and  environments  are  mimicked 
to  provide  a  realistic  operating  situation.  Ex¬ 
amples  of  live  simulations  range  from  fleet  ex¬ 
ercises  to  fire  drills. 

Eventually  live  simulations  must  be  performed 
to  validate  constructive  and  virtual  simulations. 
However,  live  simulations  are  usually  costly,  and 
trade  studies  should  be  performed  to  support  the 
balance  of  simulation  types  chosen  for  the 
program. 
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13.3  HARDWARE  VS.  SOFTWARE 

Though  current  emphasis  is  on  software  M&S, 
the  decision  of  whether  to  use  hardware,  soft¬ 
ware,  or  a  combined  approach  is  dependent  on 
the  complexity  of  the  system,  the  flexibility 
needed  for  the  simulation,  the  level  of  fidelity 
required,  and  the  potential  for  reuse.  Software 
capabilities  are  increasing,  making  software  so¬ 
lutions  cost  effective  for  large  complex  projects 
and  repeated  processes.  Hardware  methods  are 
particularly  useful  for  validation  of  software 
M&S,  simple  or  one-time  projects,  and  quick 
checks  on  changes  of  production  systems.  M&S 
methods  will  vary  widely  in  cost.  Analysis  of 
the  cost-versus-benefits  of  potential  M&S  meth¬ 
ods  should  be  performed  to  support  planning 
decisions. 


13.4  VERIFICATION,  VALIDATION, 

AND  ACCREDITATION 

How  can  you  trust  the  model  or  simulation? 
Establish  confidence  in  your  model  or  simula¬ 
tion  through  formal  verification,  validation,  and 


accreditation  (VV&A).  VV&A  is  usually  iden¬ 
tified  with  software,  but  the  basic  concept  ap¬ 
plies  to  hardware  as  well.  Figure  13-2  shows  the 
basic  differences  between  the  terms  (W&A). 

More  specifically: 

•  Verification  is  the  process  of  determining  that 
a  model  implementation  accurately  represents 
the  developer’s  conceptual  description  and 
specifications  that  the  model  was  designed  to. 

•  Validation  is  the  process  of  determining  the 
manner  and  degree  to  which  a  model  is  an 
accurate  representation  of  the  real  world  from 
the  perspective  of  the  intended  uses  of  the 
model,  and  of  establishing  the  level  of  confi¬ 
dence  that  should  be  placed  on  this  assessment. 

•  Accreditation  is  the  formal  certification  that  a 
model  or  simulation  is  acceptable  for  use  for  a 
specific  purpose.  Accreditation  is  conferred  by 
the  organization  best  positioned  to  make  the 
judgment  that  the  model  or  simulation  in  ques¬ 
tion  is  acceptable.  That  organization  may  be  an 
operational  user,  the  program  office,  or  a  con¬ 
tractor,  depending  upon  the  purposes  intended. 


Figure  13-2.  Verification,  Validation,  and  Accreditation 
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W&A  is  particularly  necessary  in  cases  where: 

•  Complex  and  critical  interoperability  is  being 
represented, 

•  Reuse  is  intended, 

•  Safety  of  life  is  involved,  and 

•  Significant  resources  are  involved. 

VV&A  Currency 

W&A  is  applied  at  initial  development  and  use. 
The  W&A  process  is  required  for  all  DoD  simu¬ 
lations  and  should  be  redone  whenever  existing 
models  and  simulations  undergo  a  major  up¬ 
grade  or  modification.  Additionally,  whenever 
the  model  or  simulation  violates  its  documented 
methodology  or  inherent  boundaries  that  were 
used  to  validate  or  verify  by  its  different  use,  then 
W&A  must  be  redone.  Accreditation,  however, 
may  remain  valid  for  the  specific  application  unless 
revoked  by  the  Accreditation  Agent,  as  long  as  its 
use  or  what  it  simulates  doesn’t  change. 

13.5  CONSIDERATIONS 

There  are  a  number  of  considerations  that  should 
enter  into  decisions  regarding  the  acquisition  and 
employment  of  modeling  and  simulation  in 
defense  acquisition  management.  Among  these  are 
such  concerns  as  cost,  fidelity,  planning,  balance, 
and  integration. 

Cost  vs.  Fidelity 

Fidelity  is  the  degree  to  which  aspects  of  the  real 
world  are  represented  in  M&S.  It  is  the  founda¬ 
tion  for  development  of  the  model  and  subsequent 
W&A.  Cost  effectiveness  is  a  serious  issue  with 
simulation  fidelity,  because  fidelity  can  be  an 
aggressive  cost  driver.  The  correct  balance  between 
cost  and  fidelity  should  be  the  result  of  simulation 
need  analysis.  M&S  designers  and  W&A  agents 
must  decide  when  enough  is  enough.  Fidelity  needs 
can  vary  throughout  the  simulation.  This  variance 
should  be  identified  by  analysis  and  planned  for. 


Note  of  caution:  Don’t  confuse  the  quality  of  the 
display  with  the  quality  of  meeting  simulation 
needs!  An  example  of  fidelity  is  a  well-known 
flight  simulator  using  a  PC  and  simple  joystick 
versus  a  full  6-degree  of  freedom  fully  instru¬ 
mented  aircraft  cockpit.  Both  have  value  at  differ¬ 
ent  stages  of  flight  training,  but  obviously  vary 
significantly  in  cost  from  thousands  of  dollars  to 
millions.  This  cost  difference  is  based  on  fidelity, 
or  degree  of  real-world  accuracy. 

Planning 

Planning  should  be  an  inherent  part  of  modeling 
and  simulation,  and,  therefore,  it  must  be  pro¬ 
active,  early,  continuous,  and  regular.  Early  plan¬ 
ning  will  help  achieve  balance  and  beneficial  re¬ 
use  and  integration.  With  computer  and  simula¬ 
tion  technologies  evolving  so  rapidly,  planning  is 
a  dynamic  process.  It  must  be  a  continuing  pro¬ 
cess,  and  it  is  important  that  the  appropriate  simu¬ 
lation  experts  be  involved  to  maximize  the  use  of 
new  capabilities.  M&S  activities  should  be  a  part 
of  the  integrated  teaming  and  involve  all  respon¬ 
sible  organizations.  Integrated  teams  must  develop 
their  M&S  plans  and  insert  them  into  the  overall 
planning  process,  including  the  TEMP,  acquisi¬ 
tion  strategy,  and  any  other  program  planning 
activity. 

M&S  planning  should  include: 

•  Identification  of  activities  responsible  for  each 
VV&A  element  of  each  model  or  simulation. 

•  Thorough  W&A  estimates,  formally  agreed 
to  by  all  activities  involved  in  M&S,  including 
T&E  commitments  from  the  developmental 
testers,  operational  testers,  and  separate  W&A 
agents. 

Those  responsible  for  the  W&A  activities  must 
be  identified  as  a  normal  part  of  planning.  Figure 
13-2  shows  the  developer  as  the  verification  agent, 
the  functional  expert  as  the  validation  agent,  and 
the  user  as  the  accreditation  agent.  In  general  this 
is  appropriate  for  virtual  simulations.  However, 
the  manufacturer  of  a  constructive  simulation 
would  usually  be  expected  to  justify  or  warran- 
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tee  their  program’s  use  for  a  particular  applica¬ 
tion.  The  question  of  who  should  actually  ac¬ 
complish  W&A  is  one  that  is  answered  in  plan¬ 
ning.  W&A  requirements  should  be  specifically 
called  out  in  tasking  documents  and  contracts. 
When  appropriate,  W&A  should  be  part  of  the 
contractor’s  proposal,  and  negotiated  prior  to  con¬ 
tract  award. 

Balance 

Balance  refers  to  the  use  of  M&S  across  the  phases 
of  the  product  life  cycle  and  across  the  spectrum 
of  functional  disciplines  involved.  The  term  may 
further  refer  to  the  use  of  hardware  vs.  software, 
fidelity  level,  W&A  level,  and  even  use  vs.  non¬ 
use.  Balance  should  always  be  based  on  cost 
effectiveness  analysis.  Cost  effectiveness  analy¬ 
ses  should  be  comprehensive;  that  is,  M&S  should 
be  properly  considered  for  use  in  all  parallel 
applications  and  across  the  complete  life  cycle  of 
the  system  development  and  use. 


Integration 

Integration  is  obtained  by  designing  a  model  or 
simulation  to  inter-operate  with  other  models  or 
simulations  for  the  purpose  of  increased  perfor¬ 
mance,  cost  benefit,  or  synergism.  Multiple  ben¬ 
efits  or  savings  can  be  gained  from  increased 
synergism  and  use  over  time  and  across  activities. 
Integration  is  achieved  through  re-use  or  upgrade 
of  legacy  programs  used  by  the  system,  or  of  the 
proactive  planning  of  integrated  development  of 
new  simulations.  In  this  case  integration  is  accom¬ 
plished  through  the  planned  utilization  of  models, 
simulations,  or  data  for  multiple  times  or  applica¬ 
tions  over  the  system  life  cycle.  The  planned  up¬ 
grade  of  M&S  for  evolving  or  parallel  uses 
supports  the  application  of  open  systems  architec¬ 
ture  to  the  system  design.  M&S  efforts  that  are 
established  to  perform  a  specific  function  by  a 
specific  contractor,  subcontractor,  or  government 
activity  will  tend  to  be  sub-optimized.  To  achieve 


Figure  13-3.  A  Robust  Integrated  Use  of  Simulation  Technology 
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integration  M&S  should  be  managed  at  least  at 
the  program  office  level. 

The  Future  Direction 

DoD,  the  Services,  and  their  commands  have 
strongly  endorsed  the  use  of  modeling  and  simu¬ 
lation  throughout  the  acquisition  life  cycle.  The 
supporting  simulation  technology  is  also  evolv¬ 
ing  as  fast  as  computer  technology  changes, 
providing  greater  fidelity  and  flexibility.  As  more 
simulations  are  interconnected,  the  opportunities 
for  further  integration  expand.  M&S  successes  to 
date  also  accelerate  its  use.  The  current  focus  is  to 
achieve  open  systems  of  simulations,  so  they  can 
be  plug-and-play  across  the  spectrum  of  applica¬ 
tions.  From  concept  analysis  through  disposal 
analysis,  programs  may  use  hundreds  of  different 
simulations,  simulators  and  model  analysis  tools. 
Figure  13-3  shows  conceptually  how  an  integrated 
program  modeling  and  simulation  would  affect  the 
functions  of  the  acquisition  process. 

A  formal  DoD  initiative,  Simulation  Based  Ac¬ 
quisition  (SBA),  is  currently  underway.  The  SBA 
vision  is  to  advance  the  implementation  of  M&S 
in  the  DoD  acquisition  process  toward  a  robust, 
collaborative  use  of  simulation  technology  that  is 
integrated  across  acquisition  phases  and  programs. 
The  result  will  be  programs  that  are  much  better 
integrated  in  an  IPPD  sense,  and  which  are  much 
more  efficient  in  the  use  of  time  and  dollars  ex¬ 
pended  to  meet  the  needs  of  operational  users. 


13.6  SUMMARY  COMMENTS 

•  Modeling  and  simulation  provides  virtual 
duplication  of  products  and  processes,  and  rep¬ 
resent  those  products  or  processes  in  readily 
available  and  operationally  valid  environments. 

•  Modeling  and  simulation  should  be  applied 
throughout  the  system  life  cycle  in  support  of 
systems  engineering  activities. 

•  The  three  classes  of  models  and  simulations 
are  virtual,  constructive,  and  live. 

•  Establish  confidence  in  your  model  or  simula¬ 
tion  through  formal  verification,  validation,  and 
accreditation. 

•  M&S  planning  should  be  an  inherent  part  of 
Systems  Engineering  planning,  and,  therefore, 
pro-active,  early,  continuous,  and  regular. 

•  A  more  detailed  discussion  of  the  use  and  man¬ 
agement  of  M&S  in  DoD  acquisition  is  avail¬ 
able  in  the  DSMC  publication  Systems  Acqui¬ 
sition  Manager’s  Guide  for  the  Use  of  Models 
and  Simulations. 

•  An  excellent  second  source  is  the  DSMC  pub¬ 
lication,  Simulation  Based  Acquisition  —A  New 
Approach.  It  surveys  applications  of  increas¬ 
ing  integration  of  simulation  in  current  DoD 
programs  and  the  resulting  increasing  benefits 
through  greater  integration. 
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14.1  METRICS  IN  MANAGEMENT 

Metrics  are  measurements  collected  for  the  pur¬ 
pose  of  determining  project  progress  and  overall 
condition  by  observing  the  change  of  the  measured 
quantity  over  time.  Management  of  technical 
activities  requires  use  of  three  basic  types  of 
metrics: 

•  Product  metrics  that  track  the  development  of 
the  product, 

•  Earned  Value  which  tracks  conformance  to  the 
planned  schedule  and  cost,  and 

•  Management  process  metrics  that  track  man¬ 
agement  activities. 

Measurement,  evaluation  and  control  of  metrics 
is  accomplished  through  a  system  of  periodic 
reporting  must  be  planned,  established,  and  moni¬ 
tored  to  assure  metrics  are  properly  measured, 
evaluated,  and  the  resulting  data  disseminated. 

Product  Metrics 

Product  metrics  are  those  that  track  key  attributes 
of  the  design  to  observe  progress  toward  meeting 
customer  requirements.  Product  metrics  reflect 
three  basic  types  of  requirements:  operational  per¬ 
formance,  life  cycle  suitability,  and  affordability. 
The  key  set  of  systems  engineering  metrics  are 
the  Technical  Performance  Measurements  (TPM.) 
TPMs  are  product  metrics  that  track  design 
progress  toward  meeting  customer  performance 
requirements.  They  are  closely  associated  with  the 
system  engineering  process  because  they  directly 
support  traceability  of  operational  needs  to  the 
design  effort.  TPMs  are  derived  from  Measures  of 
Performance  (MOPs)  which  reflect  system 
requirements.  MOPs  are  derived  from  Measures 


of  Effectiveness  (MOEs)  which  reflect  operational 
performance  requirements. 

The  term  “metric”  implies  quantitatively  measur¬ 
able  data.  In  design,  the  usefulness  of  metric  data 
is  greater  if  it  can  be  measured  at  the  configura¬ 
tion  item  level.  For  example,  weight  can  be  esti¬ 
mated  at  all  levels  of  the  WBS.  Speed,  though  an 
extremely  important  operational  parameter,  can¬ 
not  be  allocated  down  through  the  WBS.  It  cannot 
be  measured,  except  through  analysis  and  simula¬ 
tion,  until  an  integrated  product  is  available.  Since 
weight  is  an  important  factor  in  achieving  speed 
objectives,  and  weight  can  be  measured  at  various 
levels  as  the  system  is  being  developed,  weight 
may  be  the  better  choice  as  a  metric.  It  has  a  direct 
impact  on  speed,  so  it  traces  to  the  operational 
requirement.  But,  most  importantly,  it  can  be 
allocated  throughout  the  WBS,  and  progress 
toward  achieving  weight  goals,  and  then  tracked 
through  development  to  production. 

Measures  of  Effectiveness  and  Suitability 

Measures  of  Effectiveness  (MOEs)  and  Measures 
of  Suitability  (MOSs)  are  measures  of  operational 
effectiveness  and  suitability  in  terms  of  operational 
outcomes.  They  identify  the  most  critical  perfor¬ 
mance  requirements  to  meet  system-level  mission 
objectives,  and  will  reflect  key  operational  needs 
in  the  operational  requirements  document. 

Operational  effectiveness  is  the  overall  degree  of 
a  system’s  capability  to  achieve  mission  success 
considering  the  total  operational  environment.  For 
example,  weapon  system  effectiveness  considers 
environmental  factors  such  as  operator  organiza¬ 
tion,  doctrine,  and  tactics;  survivability;  vulner¬ 
ability;  and  threat  characteristics.  Measures  of 
Suitability,  on  the  other  hand,  measure  the  extent 
to  which  the  system  integrates  well  into  the 
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operation  environment  and  consider  such  issues 
as  supportability,  human  interface  compatibility, 
and  maintainability. 

Measures  of  Performance 

MOPs  characterize  physical  or  functional  attributes 
relating  to  the  execution  of  the  mission  or  func¬ 
tion.  They  quantify  a  technical  or  performance 
requirement  directly  derived  from  MOEs  and 
MOSs.  MOPs  should  relate  to  these  measures  such 
that  a  change  in  MOP  can  be  related  to  a  change 
in  MOE  or  MOS.  MOPs  should  also  reflect  key 
performance  requirements  in  the  system  specifi¬ 
cation.  MOPs  are  used  to  derive,  develop,  sup¬ 
port,  and  document  the  performance  requirements 
that  will  be  the  basis  for  design  activities  and  pro¬ 
cess  development.  They  also  identify  the  critical 
technical  parameters  that  will  be  tracked  through 
Technical  Performance  Measurements. 

Technical  Performance  Measurements 

TPMs  are  derived  directly  from  MOPs,  and  are 
selected  as  being  critical  from  a  periodic  review 
and  control  standpoint.  TPMs  help  assess  design 
progress,  assess  compliance  to  requirements 
throughout  the  WBS,  and  assist  in  monitoring  and 
tracking  technical  risk.  They  can  identify  the  need 
for  deficiency  recovery,  and  provide  information 
to  support  cost-performance  sensitivity  assess¬ 
ments.  TPMs  can  include  range,  accuracy,  weight, 
size,  availability,  power  output,  power  required, 
process  time,  and  other  product  characteristics 
that  relate  directly  to  the  system  operational 
requirements. 

TPMs  traceable  to  WBS  elements  are  preferred, 
so  elements  within  the  system  can  be  monitored 
as  well  as  the  system  as  a  whole.  However,  some 
necessary  TPMs  will  be  limited  to  the  system  or 
subsystem  level.  For  example,  the  specific  fuel 
consumption  of  an  engine  would  be  a  TPM  neces¬ 
sary  to  track  during  the  engine  development,  but 
it  is  not  allocated  throughout  the  WBS.  It  is 
reported  as  a  single  data  item  reflecting  the  per¬ 
formance  of  the  engine  as  a  whole.  In  this  case  the 
metric  will  indicate  that  the  design  approach  is 
consistent  with  the  required  performance,  but  it 


may  not  be  useful  as  an  early  warning  device  to 
indicate  progress  toward  meeting  the  design  goal. 
Additional  information  on  the  TPM  concept  is 
provided  at  the  end  of  this  chapter. 

Example  of  Measures 

MOE:  The  vehicle  must  be  able  to  drive  fully 
loaded  from  Washington,  DC  to  Tampa,  Florida 
on  one  tank  of  fuel. 

MOP:  Vehicle  range  must  be  equal  to  or  greater 
than  1000  miles. 

TPM:  Fuel  consumption,  vehicle  weight,  tank  size, 
drag,  power  train  friction,  etc. 

Suitability  Metrics 

Tracking  metrics  relating  to  operational  suitabil¬ 
ity  and  other  life  cycle  concerns  may  be  appropri¬ 
ate  to  monitor  progress  toward  an  integrated 
design.  Operational  suitability  is  the  degree  to 
which  a  system  can  be  placed  satisfactorily  in  field 
use  considering:  availability,  compatibility,  trans¬ 
portability,  interoperability,  reliability,  usage  rates, 
maintainability,  safety,  human  factors,  documen¬ 
tation,  training,  manpower,  supportability,  logis¬ 
tics,  and  environmental  impacts.  These  suitability 
parameters  can  generate  product  metrics  that  indi¬ 
cate  progress  toward  an  operationally  suitable 
system.  For  example,  factors  that  indicate  the  level 
of  automation  in  the  design  would  reflect  progress 
toward  achieving  manpower  quantity  and  quality 
requirements.  TPMs  and  suitability  product 
metrics  commonly  overlap.  For  example,  Mean 
Time  Between  Failure  (MBTF)  can  reflect  both 
effectiveness  or  suitability  requirements. 

Suitability  metrics  would  also  include  measure¬ 
ments  that  indicate  improvement  in  the 
producibility,  testability,  degree  of  design  sim¬ 
plicity,  and  design  robustness.  For  example, 
tracking  number  of  parts,  number  of  like-parts, 
and  number  of  wearing  parts  provides  indicators 
of  producibility,  maintainability,  and  design 
simplicity. 
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Product  Affordability  Metrics 

Estimated  unit  production  cost  can  be  tracked  dur¬ 
ing  the  design  effort  in  a  manner  similar  to  the 
TPM  approach,  with  each  Cl  element  reporting 
an  estimate  based  on  current  design.  These  esti¬ 
mates  are  combined  at  higher  WBS  levels  to  pro¬ 
vide  subsystem  and  system  cost  estimates.  This 
provides  a  running  engineering  estimate  of  unit 
production  cost,  tracking  of  conformance  to  Design 
to  Cost  (DTC)  goals,  and  a  method  to  isolate  design 
problems  relating  to  production  costs. 

Lifecycle  affordability  can  be  tracked  through 
factors  that  are  significant  in  parametric  life  cycle 
cost  calculations  for  the  particular  system.  For 
example,  two  factors  that  reflect  life  cycle  cost  for 
most  transport  systems  are  fuel  consumption  and 
weight,  both  of  which  can  be  tracked  as  metrics. 

Timing 

Product  metrics  are  tied  directly  to  the  design  pro¬ 
cess.  Planning  for  metric  identification,  reporting, 
and  analysis  is  begun  with  initial  planning  in  the 
concept  exploration  phase.  The  earliest  systems 
engineering  planning  should  define  the  manage¬ 
ment  approach,  identify  performance  or  charac¬ 
teristics  to  be  measured  and  tracked,  forecast  values 
for  those  performances  or  characteristics,  deter¬ 
mine  when  assessments  will  be  done,  and  establish 
the  objectives  of  assessment. 

Implementation  is  begun  with  the  development  of 
the  functional  baseline.  During  this  period,  sys¬ 
tems  engineering  planning  will  identify  critical 
technical  parameters,  time  phase  planned  profiles 
with  tolerance  bands  and  thresholds,  reviews  or 
audits  or  events  dependent  or  critical  for  achieve¬ 
ment  of  planned  profiles,  and  the  method  of  esti¬ 
mation.  During  the  design  effort,  from  functional 
to  product  baseline,  the  plan  will  be  implemented 
and  continually  updated  by  the  systems  engineer¬ 
ing  process.  To  support  implementation,  contracts 
should  include  provision  for  contractors  to  provide 
measurement,  analysis,  and  reporting.  The  need 
to  track  product  metrics  ends  in  the  production 
phase,  usually  concurrent  with  the  establishment 
of  the  product  (as-built)  baseline. 


DoD  and  Industry  Policy  on  Product  Metrics 

The  establishment  of  performance  metrics  to 
provide  measures  of  how  well  the  technical  devel¬ 
opment  and  design  are  evolving  relative  to  what 
was  planned  and  relative  to  meeting  system 
requirements  in  terms  of  performance,  risk  miti¬ 
gation,  producibility,  cost  and  schedule.  Perfor¬ 
mance  metrics  must  be  traceable  to  performance 
parameters  identified  by  the  operational  user.  DoD 
5000.2-R,  Part  4,  Par.  4.3. 

The  performing  activity  establishes  and  imple¬ 
ments  TPM  to  evaluate  the  adequacy  of  evolving 
solutions  to  identify  deficiencies  impacting  the 
ability  of  the  system  to  satisfy  a  designated  value 
for  a  technical  parameter.  ELA  IS-632,  Section  3. 

The  performing  activity  identifies  the  technical 
performance  measures  which  are  key  indicators 
of  system  performance  ....  should  be  limited  to 
critical  MOPs  which,  if  not  met  put  the  project  at 
cost,  schedule,  or  performance  risk.  IEEE  1220, 
Section  6. 


14.2  EARNED  VALUE 

Earned  Value  is  a  metric  reporting  system  that  uses 
cost-performance  metrics  to  track  the  cost  and 
schedule  progress  of  system  development  against 
a  projected  baseline.  It  is  a  “big  picture”  approach 
and  integrates  concerns  related  to  performance, 
cost,  and  schedule.  As  shown  by  Figure  14-1, 
earned  value  warns  of  schedule  and  cost  problems 
that  are  based  on  variance  from  a  projected  per¬ 
formance.  The  projected  performance  is  based  on 
estimates  of  appropriate  cost  and  schedule  to 
perform  the  work  required  by  each  WBS  element. 
When  a  variance  occurs  the  system  engineer  can 
pinpoint  WBS  elements  that  have  potential  tech¬ 
nical  development  problems.  Combined  with  prod¬ 
uct  metrics,  earned  value  is  a  powerful  technical 
management  tool  for  detecting  and  understanding 
development  problems. 

Relationships  exist  between  product  metrics,  the 
event  schedule,  the  calendar  schedule,  and  Earned 
Value: 
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Figure  14-1.  Earned  Value  Concept 


> 


•  The  Event  Schedule  includes  tasks  for  each 
event/exit  criteria  that  must  be  performed  to 
meet  key  system  requirements,  which  are 
directly  related  to  product  metrics. 

•  The  Calendar  (Detail)  Schedule  includes  time 
frames  established  to  meet  those  same  product 
metric  related  objectives  (schedules). 

•  Earned  Value  includes  cost/schedule  impacts 
of  not  meeting  those  objectives,  and  when 
correlated  with  product  metrics  can  identify 
emerging  program  and  technical  risk. 

14.3  PROCESS  METRICS 

Management  process  metrics  are  measurements 
taken  to  track  the  process  of  developing,  build¬ 
ing,  and  introducing  the  system.  They  include 
a  wide  range  of  potential  factors  and  selection 
is  program-unique.  They  measure  such  factors 
as  availability  of  resources,  activity  time  rates, 
items  completed,  completion  rates,  and  customer 
or  team  satisfaction. 


Examples  of  these  factors  are:  number  of  trained 
personnel  onboard,  average  time  to  approve/ 
disapprove  ECPs,  lines  of  code  or  drawings 
released,  ECPs  resolved  per  month,  and  team  risk 
identification  or  feedback  assessments.  Selection 
of  appropriate  metrics  should  be  done  to  track  key 
management  activities.  Selection  of  these  metrics 
is  part  of  the  systems  engineering  planning  process. 

How  Much  Metrics? 

The  choice  of  the  amount  and  depth  of  metrics  is 
a  planning  function  that  seeks  a  balance  between 
risk  and  cost.  It  depends  on  many  considerations, 
including  system  complexity,  organizational  com¬ 
plexity,  reporting  frequency,  how  many  contrac¬ 
tors,  program  office  size  and  make-up,  contractor 
past  performance,  political  visibility,  and  contract 
type. 

14.4  SUMMARY  POINTS 

•  Management  of  technical  activities  requires  use 
of  three  basic  types  of  metrics:  product  metrics 
that  track  the  development  of  the  product, 
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earned  value  which  tracks  conformance  to 
the  planned  schedule  and  cost,  and  manage¬ 
ment  process  metrics  that  track  management 
activities. 

•  Measurement,  evaluation  and  control  of  metrics 
is  accomplished  through  a  system  of  periodic 
reporting  that  must  be  planned,  established,  and 
monitored  to  ensure  that  metrics  are  measured 
properly,  evaluated,  and  the  resulting  data 
disseminated. 


•  TPMs  are  performance  based  product  metrics 
that  track  progress  through  measurement  of  key 
technical  parameters.  They  are  important  to  the 
systems  engineering  process  because  they 
connect  operational  requirements  to  measurable 
design  characteristics  and  help  assess  how  well 
the  effort  is  meeting  those  requirements.  TPMs 
are  required  for  all  programs  covered  by  DoD 
5000.2-R. 
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A  TPM  should  be  a  significant  qualifier  of  the  total 
system,  and  reflect  a  characteristic  that  contrib¬ 
utes  to  system  success.  Critical  technical  param¬ 
eters  can  be  derived  from  risk  issues,  system  quali¬ 
fiers,  safety  issues,  cost/schedule  drivers,  mission 
critical  issues,  and  contract  performance  concerns. 

A  TPM  is  managed  by  exception.  If  the  reported 
measurement  is  outside  a  set  of  acceptable  values, 
then  the  metric  indicates  a  problem  is  occurring. 


TPM  reporting  is  analogous  to  a  smoke  alarm;  if 
the  metric  is  within  bounds  there  is  no  alert.  TPM 
reporting  flags  design  deficiencies  or  excesses,  and 
permits  timely  action  to  correct  them. 

TPM  profiles  are  developed  from  historical  data, 
test  data,  system  engineering  planning  estimates, 
and  contract  requirements.  A  typical  profile  is 
shown  in  Figure  14-2. 


Achievement  to  date  -  Measured  or  estimated  progress  plotted  and  compared  with  planned  progress  by  designated  milestone 
date. 

Current  estimate  -  Expected  value  of  a  technical  parameter  at  contract  completion. 

Planned  value  -  Predicted  value  of  parameter  at  a  given  point  in  time. 

Planned  profile  -  Time  phased  projected  planned  values. 

Tolerance  band  -  Management  alert  limits  representing  projected  level  of  estimating  error. 

Threshold  -  Limiting  acceptable  value,  usually  contractual. 

Variance  -  Difference  between  the  planned  value  and  the  achievement-to-date  derived  from  analysis,  test,  or  demon¬ 
stration. 


Figure  14-2.  Technical  Performance  Measurement  -  The  Concept 
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15.1  RISK  AS  REALITY 

Risk  is  inherent  in  all  activities.  It  is  a  normal 
condition  of  existence.  Risk  is  the  potential  for  a 
negative  future  reality  that  may  or  may  not  hap¬ 
pen.  Risk  is  defined  by  two  characteristics  of  a 
possible  negative  future  event:  probability  of 
occurrence  (whether  something  will  happen),  and 
consequences  of  occurrence  (how  catastrophic  if 
it  happens).  If  the  probability  of  occurrence  is  not 
known,  then  one  has  uncertainty ,  and  the  risk  is 
undefined. 

Risk  is  not  a  problem.  It  is  an  understanding  of 
the  level  of  threat  due  to  potential  problems.  A 
problem  is  a  consequence  that  has  already 
occurred.  In  fact,  knowledge  of  a  risk  is  an  oppor¬ 
tunity  to  avoid  a  problem.  Risk  occurs  whether 
there  is  an  attempt  to  manage  it  or  not. 


Risk  exists  whether  you  acknowledge  it,  whether 
you  believe  it,  whether  it  is  written  down,  or 
whether  you  understand  it.  Risk  does  not  change 
because  you  hope  it  will,  you  ignore  it,  or  your 
boss’s  expectations  do  not  reflect  it.  Nor  will  it 
change  just  because  it  is  contrary  to  policy,  proce¬ 
dure,  or  regulation.  Risk  is  neither  good  nor  bad. 
It  is  just  how  things  are.  Progress  and  opportunity 
are  companions  of  risk.  In  order  to  make  progress, 
risks  must  be  understood,  managed,  and  reduced 
to  acceptable  levels. 

Types  of  Risk  in  a 

Systems  Engineering  Environment 

Systems  engineering  management  related  risks 
could  be  related  to  the  system  products  or  to  the 
process  of  developing  the  system.  Figure  15-1 
shows  the  decomposition  of  system  development 
risks. 


Figure  15-1.  Risk  Hierarchy 
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Risks  related  to  system  development  generally  are  events.  Risk  management  depends  on  risk  man- 
traceable  to  achieving  life  cycle  customer  require-  agement  planning,  early  identification  and  analy- 
ments.  Product  risks  include  both  end  product  risks  sis  of  risks,  continuous  risk  tracking  and  reassess- 
that  relate  to  the  basic  performance  and  cost  of  the  ment,  early  implementation  of  corrective  actions, 
system,  and  to  enabling  products  that  relate  to  the  communication,  documentation,  and  coordination, 
products  that  produce,  maintain,  support,  test,  train,  Though  there  are  many  ways  to  structure  risk 
and  dispose  of  the  system.  management,  this  book  will  structure  it  as  having 

four  parts:  Planning,  Assessment,  Handling,  and 
Risks  relating  to  the  management  of  the  develop-  Monitoring.  As  depicted  in  Figure  15-2  all  of  the 

ment  effort  can  be  technical  management  risk  or  parts  are  interlocked  to  demonstrate  that  after  initial 

risk  caused  by  external  influences.  Risks  dealing  planning  the  parts  begin  to  be  dependent  on  each 

with  the  internal  technical  management  include  other.  Illustrating  this,  Figure  15-3  shows  the  key 

those  associated  with  schedules,  resources,  work  control  and  feedback  relationships  in  the  process, 

flow,  on  time  deliverables,  availability  of  appro¬ 
priate  personnel,  potential  bottlenecks,  critical  path  Risk  Planning 
operations,  and  similar.  Risks  dealing  with  exter¬ 
nal  influences  include  resource  availability,  higher  Risk  Planning  is  the  continuing  process  of  devel- 

authority  delegation,  level  of  program  visibility,  oping  an  organized,  comprehensive  approach  to 

regulatory  requirements,  and  the  like.  risk  management.  The  initial  planning  includes 

establishing  a  strategy;  establishing  goals  and 
objectives;  planning  assessment,  handling,  and 
15.2  RISK  MANAGEMENT  monitoring  activities;  identifying  resources,  tasks, 

and  responsibilities;  organizing  and  training  risk 
Risk  management  is  an  organized  method  for  management  IPT  members;  establishing  a  method 

identifying  and  measuring  risk  and  for  selecting,  to  track  risk  items;  and  establishing  a  method  to 

developing,  and  implementing  options  for  the  document  and  disseminate  information  on  a 

handling  of  risk.  It  is  a  process,  not  a  series  of  continuous  basis. 


Figure  15-2.  Four  Elements  of  Risk  Management 
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Feedback  for 
Management 


Figure  15-3.  Risk  Management  Control  and  Feedback 


In  a  systems  engineering  environment,  risk 
planning  should  be: 

•  Inherent  (imbedded)  in  systems  engineering 
planning  and  other  related  planning,  such  as 
producibility,  supportability,  and  configuration 
management; 

•  A  documented  continuous  effort; 

•  Integrated  among  all  activities; 

•  Integrated  with  other  planning,  such  as  systems 
engineering  planning,  supportability  analysis, 
production  planning,  configuration  and  data 
management,  etc.; 

•  Integrated  with  previous  and  future  phases;  and 

•  Selected  for  each  Configuration  Baseline. 

Risk  is  altered  by  time.  As  we  try  to  control  or 
alter  risk,  its  probability  and/or  consequence  will 
change.  Judgment  of  the  risk  impact  and  the 
method  of  handling  the  risk  must  be  reassessed 


and  potentially  altered  as  events  unfold.  Since  these 
events  are  continually  changing,  the  planning 
process  is  a  continuous  one. 

Risk  Assessment 

Risk  assessment  consists  of  identifying  and  ana¬ 
lyzing  the  risks  associated  with  the  life  cycle  of 
the  system. 

Risk  Identification  Activities 

Risk  identification  activities  establish  what  risks 
are  of  concern.  These  activities  include: 

•  Identifying  risk/uncertainty  sources  and  drivers, 

•  Transforming  uncertainty  into  risk, 

•  Quantifying  risk, 

•  Establishing  probability,  and 

•  Establishing  the  priority  of  risk  items. 
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As  shown  by  Figure  15-4  the  initial  identification 
process  starts  with  an  identification  of  potential 
risk  items  in  each  of  the  four  risk  areas.  Risks 
related  to  the  system  performance  and  supporting 
products  are  generally  organized  by  work  break¬ 
down  structure  and  initially  determined  by  expert 
assessment  of  teams  and  individuals  in  the  devel¬ 
opment  enterprise.  These  risks  tend  to  be  those 
that  require  follow-up  quantitative  assessment. 
Internal  process  and  external  influence  risks  are 
also  determined  by  expert  assessment  within  the 
enterprise,  as  well  as  through  the  use  of  risk  area 
templates  similar  to  those  found  in  DoD  4245.7- 
M.  The  DoD  4245.7-M  templates  describe  the  risk 
areas  associated  with  system  acquisition  manage¬ 
ment  processes,  and  provide  methods  for  reduc¬ 
ing  traditional  risks  in  each  area.  These  templates 
should  be  tailored  for  specific  program  use  based 
on  expert  feedback. 

After  identifying  the  risk  items,  the  risk  level 
should  be  established.  One  common  method  is 
through  the  use  of  a  matrix  such  as  shown  in  Figure 
15-5.  Each  item  is  associated  with  a  block  in  the 


matrix  to  establish  relative  risk  among  them.  On 
such  a  graph,  risk  increases  on  the  diagonal  and 
provides  a  method  for  assessing  relative  risk.  Once 
the  relative  risk  is  known,  a  priority  list  can  be 
established  and  risk  analysis  can  begin. 

Risk  identification  efforts  can  also  include  activi¬ 
ties  that  help  define  the  probability  or  consequences 
of  a  risk  item,  such  as: 

•  Testing  and  analyzing  uncertainty  away, 

•  Testing  to  understand  probability  and 
consequences, 

•  Activities  that  quantify  risk  where  the  qualita¬ 
tive  nature  of  High,  Moderate,  Low  estimates 
are  insufficient  for  adequate  understanding. 

Risk  Analysis  Activities 

Risk  analysis  activities  continue  the  assessment 
process  by  refining  the  description  of  identified 
risk  event  through  isolation  of  the  cause  of  risk, 
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Figure  15-5.  Simple  Risk  Matrix 
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determination  of  the  full  impact  of  risk,  and  the 
determination  and  choose  of  alternative  courses 
of  action.  They  are  used  to  determine  what  risk 
should  be  tracked,  what  data  is  used  to  track  risk, 
and  what  methods  are  used  to  handle  the  risk. 

Risk  analysis  explores  the  options,  opportunities, 
and  alternatives  associated  with  the  risk.  It 
addresses  the  questions  of  how  many  legitimate 
ways  the  risk  could  be  dealt  with  and  the  best  way 
to  do  so.  It  examines  sensitivity,  and  risk  interre¬ 
lationships  by  analyzing  impacts  and  sensitivity 
of  related  risks  and  performance  variation.  It  further 
analyzes  the  impact  of  potential  and  accomplished, 
external  and  internal  changes. 

Risk  analysis  activities  that  help  define  the  scope 
and  sensitivity  of  the  risk  item  include  finding 
answers  to  the  following  questions: 

•  If  something  changes,  will  risk  change  faster, 
slower,  or  at  the  same  pace? 

•  If  a  given  risk  item  occurs,  what  collateral  ef¬ 
fects  happen? 

•  How  does  it  affect  other  risks? 


•  How  does  it  affect  the  overall  situation? 

•  Development  of  a  watch  list  (prioritized  list  of 
risk  items  that  demand  constant  attention  by 
management)  and  a  set  of  metrics  to  determine 
if  risks  are  steady,  increasing,  or  decreasing. 

•  Development  of  a  feedback  system  to  track 
metrics  and  other  risk  management  data. 

•  Development  of  quantified  risk  assessment. 

Quantified  risk  assessment  is  a  formal  quantifica¬ 
tion  of  probabilities  of  occurrence  and  conse¬ 
quences  using  a  top-down  structured  process 
following  the  work  breakdown  structure.  For  each 
element,  risks  are  assessed  through  analysis,  simu¬ 
lation  and  test  to  determine  statistical  probability 
and  specific  conditions  caused  by  the  occurrence 
of  the  consequence. 

Cautions  in  Risk  Assessments 

Reliance  solely  on  numerical  values  from  simula¬ 
tions  and  analysis  should  be  avoided.  Do  not  lose 
sight  of  the  actual  source  and  consequences  of  the 
risks.  Testing  does  not  eliminate  risk.  It  only 
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provides  data  to  assess  and  analyze  risk.  Most  of 
all,  beware  of  manipulating  relative  numbers,  such 
as  ‘risk  index”  or  “risk  scales,”  even  when  based 
on  expert  opinion,  as  quantified  data.  They  are 
important  pieces  of  information,  but  they  are  largely 
subjective  and  relative;  they  do  not  necessarily 
define  risk  accurately.  Numbers  such  as  these  should 
always  be  the  subject  of  a  sensitivity  analysis. 

Risk  Handling 

Once  the  risks  have  been  categorized  and  analyzed, 
the  process  of  handling  those  risks  is  initiated.  The 
prime  purpose  of  risk  handling  activities  is  to  miti¬ 
gate  risk.  Methods  for  doing  this  are  numerous, 
but  all  fall  into  four  basic  categories: 

•  Risk  Avoidance, 

•  Risk  Control, 

•  Risk  Assumption,  and 

•  Risk  Transfer 
Avoidance 

To  avoid  risk,  remove  requirements  that  represent 
uncertainty  and  high  risk  (probability  or  conse¬ 
quence).  Avoidance  includes  trading  off  risk  for 
performance  or  other  capability,  and  it  is  a  key 
activity  during  requirements  analysis.  Avoidance 
requires  understanding  of  priorities  in  requirements 
and  constraints.  Are  they  mission  critical,  mission 
enhancing,  “bells  and  whistles,”  or  nice  to  have? 

Control 

Control  is  the  deliberate  use  of  the  design  process 
to  lower  the  risk  to  acceptable  levels.  It  requires 
the  disciplined  application  of  the  systems  engi¬ 
neering  process  and  detailed  knowledge  of  the 
technical  area  associated  with  the  design.  Control 
techniques  are  plentiful  and  include: 

•  Multiple  concurrent  design  to  provide  more 
than  one  design  path  to  a  solution; 

•  Alternative  low-risk  design  to  minimize  the  risk 
of  a  design  solution  by  using  the  lowest  risk 
design  option; 


•  Incremental  development,  such  as  preplanned 
product  improvement,  to  disassociate  the  design 
from  high-risk  components  that  can  be 
developed  separately; 

•  Technology  maturation  that  allows  high-risk 
components  to  be  developed  separately  while 
the  basic  development  uses  a  less  risky  and 
lower  performance  temporary  substitute; 

•  Test,  analyze  and  fix  that  allows  understanding 
to  lead  to  lower  risk  design  changes.  (Test  can 
be  replaced  by  demonstration,  inspection,  early 
prototyping,  reviews,  metric  tracking,  experi¬ 
mentation,  models  and  mockups,  simulation, 
or  any  other  input  or  set  of  inputs  that  gives  a 
better  understanding  of  the  risk); 

•  Robust  design  that  produces  a  design  with  sub¬ 
stantial  margin  such  that  risk  is  reduced;  and 

•  The  open  system  approach  that  emphasizes  use 
of  generally  accepted  interface  standards  that 
provide  proven  solutions  to  component  design 
problems. 

Acceptance 

Acceptance  is  the  deliberate  assumption  of  risk 
because  it  is  low  enough  in  probability  and/or  con¬ 
sequence  to  be  reasonably  absorbed  without 
impacting  the  development  effort.  Key  techniques 
for  handling  accepted  risk  are  budget  and  sched¬ 
ule  reserves  for  unplanned  activities  and  continu¬ 
ous  assessment  (to  ensure  that  accepted  risks  are 
maintained  at  acceptance  level).  The  basic  ob¬ 
jective  of  risk  management  in  systems  engineer¬ 
ing  is  to  reduce  all  risk  to  an  acceptable  level. 

The  strong  budgetary  strain  and  tight  schedules 
on  DoD  programs  tends  to  reduce  the  program 
manager’s  and  system  engineer’s  capability  to 
provide  reserve.  By  identifying  a  risk  as  accept¬ 
able,  the  worst  case  outcome  is  being  declared 
acceptable.  Accordingly,  the  level  of  risk  consid¬ 
ered  acceptable  should  be  chosen  very  carefully 
in  a  DOD  acquisition  program. 
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Transfer 

Transfer  can  be  used  to  reduce  risk  by  moving 
the  risk  from  one  area  of  design  to  another  where 
a  design  solution  is  less  risky.  Examples  of  this 
include: 

•  Assignment  to  hardware  (vs.  software)  or  vice 
versa,  and 

•  Use  of  functional  partitioning  to  allocate 
performance  based  on  risk  factors. 

Transfer  is  most  associated  with  the  act  of  assign¬ 
ing,  delegating,  or  paying  someone  to  assume  the 
risk.  To  some  extent,  transfer  always  occurs  when 
contracting  or  tasking  another  activity.  The  con¬ 
tract  or  tasking  document  sets  up  agreements  that 
transfer  risk  from  the  government  to  contractor, 
program  office  to  agency,  and  vice  versa.  Typical 
methods  include  insurance,  warranties,  and  incen¬ 
tive  clauses.  Risk  is  never  truly  transferred.  If  the 
risk  isn’t  mitigated  by  the  delegated  activity  it  still 
affects  your  project  or  program. 

Key  areas  to  review  before  using  transfer  are: 

•  How  well  can  the  delegated  activity  handle  the 
risk?  Transfer  is  effective  only  to  the  level  the 
risk  taker  can  handle  it. 

•  How  well  will  the  delegated  activity  solution 
integrate  into  your  project  or  program?  Trans¬ 
fer  is  effective  only  if  the  method  is  integrated 
with  the  overall  effort.  For  example,  is  the  war¬ 
ranty  action  coordinated  with  operators  and 
maintainers? 

•  Was  the  method  of  tasking  the  delegated  activ¬ 
ity  proper?  Transfer  is  effective  only  if  the  trans¬ 
fer  mechanism  is  valid.  For  example,  can  in¬ 
centives  be  “gamed?” 

•  Who  has  the  most  control  over  the  risk?  If  the 
project  or  program  has  no  or  little  control  over 
the  risk  item,  then  transfer  should  be  consid¬ 
ered  to  delegate  the  risk  to  those  most  likely  to 
be  able  to  control  it. 


Monitoring  and  Reporting 

Risk  monitoring  is  the  continuous  process  of  track¬ 
ing  and  evaluating  the  risk  management  process 
by  metric  reporting,  enterprise  feedback  on  watch 
list  items,  and  regular  enterprise  input  on  poten¬ 
tial  developing  risks.  (The  metrics,  watch  lists,  and 
feedback  system  are  developed  and  maintained  as 
an  assessment  activity.)  The  output  of  this  process 
is  then  distributed  throughout  the  enterprise,  so 
that  all  those  involved  with  the  program  are  aware 
of  the  risks  that  affect  their  efforts  and  the  system 
development  as  a  whole. 

Special  Case  -  Integration  as  Risk 

Integration  of  technologies  in  a  complex  system 
is  a  technology  in  itself!  Technology  integration 
during  design  may  be  a  high-risk  item.  It  is  not 
normally  assessed  or  analyzed  as  a  separately 
identified  risk  item.  If  integration  risks  are  not 
properly  identified  during  development  of  the 
functional  baseline,  they  will  demonstrate  them¬ 
selves  as  serious  problems  in  the  development  of 
the  product  baseline. 

Special  Case  -  Software  Risk 

Based  on  past  history,  software  development  is 
often  a  high-risk  area.  Among  the  causes  of  per¬ 
formance,  schedule,  and  cost  deficiencies  have 
been: 

•  Imperfect  understanding  of  operational  require¬ 
ments  and  its  translation  into  source  instruc¬ 
tions, 

•  Risk  tracking  and  handling, 

•  Insufficient  comprehension  of  interface  con¬ 
straints,  and 

•  Lack  of  sufficient  qualified  personnel. 

Risk  Awareness 

All  members  of  the  enterprise  developing  the 
system  must  understand  the  need  to  pay  attention 
to  the  existence  and  changing  nature  of  risk. 
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Consequences  that  are  unanticipated  can  seri¬ 
ously  disrupt  a  development  effort.  The  uneasy 
feeling  that  something  is  wrong,  despite  assur¬ 
ances  that  all  is  fine  may  be  valid.  These  kinds 
of  intuitions  have  allowed  humanity  to  survive 
the  slings  and  arrows  of  outrageous  fortune 
throughout  history.  Though  generally  viewed  as 
non-analytical,  these  apprehensions  should  not 
be  ignored.  Experience  indicates  those  non-spe¬ 
cific  warnings  have  validity,  and  should  be  quan¬ 
tified  as  soon  as  possible. 


15.3  SUMMARY  POINTS 

•  Risk  is  inherent  in  all  activities. 

•  Risk  is  composed  of  knowledge  of  two  charac¬ 
teristics  of  a  possible,  negative  future  event: 
probability  of  occurrence  and  consequences  of 
occurrence. 


•  Risk  management  is  associated  with  a  clear 
understanding  of  probability. 

•  Risk  management  is  an  essential  and  integral 
part  of  technical  program  management  (systems 
engineering). 

•  Risks  and  uncertainties  must  be  identified, 
analyzed,  handled,  and  tracked. 

•  There  are  four  basic  ways  of  handling  risk: 
avoidance,  transfer,  acceptance,  and  control. 

•  Program  risks  are  classified  as  low,  moderate, 
or  high  depending  on  consequences  and  prob¬ 
ability  of  occurrence.  Risk  classification  should 
be  based  on  quantified  data  to  the  extent 
possible. 
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SUPPLEMENT  A 

RISK  MANAGEMENT 
IN  DOD  ACQUISITION 


Policy 

DoD  policy  is  quite  clear  in  regard  to  risk  man¬ 
agement:  it  must  be  done. 

The  Program  Manager  and  other  acquisition 
managers  shall  continually  assess  program  risks. 
(DoDD  5000.1.) 

The  PM  shall  establish  a  risk  management  pro¬ 
gram  for  each  acquisition  program  to  identify  and 
control  performance,  cost,  and  schedule  risks. 
(DoD  5000.2-R.) 

In  addition,  DoDD  5000.4  identifies  risk  and  cost 
analysis  as  a  PM  responsibility. 

Risk  Management  View 

A  DSMC  study  indicates  that  major  programs 
declared  moderate  risk  at  Milestone  II  have  been 
more  successful  in  terms  of  meeting  cost  and  sched¬ 
ule  goals  than  those  declared  low  risk  (DSMC  TR 
2-95.)  This  strongly  implies  that  program  offices 
that  understand  and  respect  risk  management  will 
be  more  successful.  For  this  reason  the  program 
office  needs  to  understand  a  systems  level  view  of 
risk.  The  systems  engineer  provides  this  view. 
Systems  Engineering  is  the  foundation  of  program 
office  risk  assessments  because  it  is  the  connec¬ 
tion  to  the  reality  of  system  development  and 
production,  the  program  office’s  primary  mission. 

However,  the  program  office  has  external  risks 
to  deal  with  as  well  as  the  internal  risks  preva¬ 
lent  in  the  development  process.  The  Systems 
Engineer  has  to  provide  the  Program  Manager 
internal  risk  data  in  a  manner  that  aids  the  han¬ 
dling  of  the  external  risks.  In  short,  the  systems 


engineer  must  present  bad  news  such  that  it  is 
reasonable  and  compelling  to  higher  levels  of 
authority. 

Factoring  Risk  Management  into  the  Process 

Risk  management,  as  an  integral  part  of  the  over¬ 
all  program  planning  and  management  process,  is 
enhanced  by  applying  a  controlled,  consistent, 
approach  to  systems  engineering  and  using 
integrated  teams  for  both  product  development  and 
management  control.  Programs  should  be 
transitioned  to  the  next  phase  only  if  risk  assess¬ 
ment  determines  risk  is  at  the  appropriate  level. 
Know  the  risk  drivers  behind  the  estimates.  By  its 
nature  there  are  subjective  aspects  of  assessing  and 
analyzing  risk  at  the  system  level,  even  though 
they  tend  to  be  represented  as  quantitative  and/or 
analytically  objective. 

Risk  and  Phases 

During  Concept  Exploration  initial  system-level 
risk  assessments  are  made.  Unknown-unknowns, 
uncertainty,  and  some  high  risk  elements  are 
normal  and  expected. 

Program  Definition  and  Risk  Reduction  (PDRR) 
is  a  major  technology  risk  reduction  effort.  Its 
purpose  is  to  identify  and  reduce  technical  risk  to 
level  necessary  to  allow  engineering  development 
of  the  system.  PDRR  risk  efforts  emphasize: 

•  Testing,  analyzing,  or  mitigating  system  and 
subsystem  uncertainty  and  high  risk  out  of  the 
program. 

•  Demonstrating  technology  sufficient  to  uncover 
system  and  subsystem  unknown-unknowns 
(especially  for  integration). 
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•  Planning  for  risk  management  in  Engineering 
and  Manufacturing  Development  (EMD), 
especially  handling  of  moderate  risk  and 
tracking  of  risk. 

EMD  requires  the  application  of  product  and 
manufacturing  engineering,  which  can  be  dis¬ 
rupted  if  the  technology  development  is  not 
sufficient  to  support  engineering  development. 
Risk  management  in  EMD  emphasizes: 

•  Reduction  and  control  of  moderate  risks, 

•  All  risks  under  management  including 
emerging  ones,  and 

•  Maintenance  of  risk  levels  and  reaction  to 
problems. 

Key  to  Program  Success:  PDRR 

PDRR  is  the  phase  designed  to  reduce  technical 
risk  to  a  level  manageable  by  engineers — that  is, 
the  technology  is  developed,  understood,  and  is 
malleable  by  engineering  processes.  If  it  is  not, 
program  disruption  will  occur. 

Focus  on  lowering  the  risk  associated  with  tech¬ 
nology  maturity  is  essential  in  PDRR.  Doing  it 
later  carries  the  dual  risks  of  concurrency  and  un¬ 
realistic  management  expectations.  Tasking,  con¬ 
tract  requirements,  and  management  expectations 
in  EMD  are  based  on  an  understanding  established 
at  MSII  that  the  system  is  ready  for  engineering 
development.  After  PDRR  the  acquisition  com¬ 
munity  generally  assumes  that  risk  is  moderate  to 
low,  that  the  technology  is  “available.”  Experience 
shows  us  that  technology  integration,  both  inter¬ 
nal  and  external,  and  software  are  risk  areas  that 
tend  to  be  high  and  should  be  rigorously  addressed 
in  PDRR.  Risk  reduction  in  these  areas  tends  to 
be  expensive  (e.g.,  prototyping)  or  dependent  on 
external  drivers  (e.g.,  C4I).  During  PDRR  cost 
trade-off  with  major  technical  risk  efforts  is 
dangerous,  but  has  been  common  to  see  PDRR 
under-funded. 


Risk  Classification  on  the 
System  (Program)  Level 

Classification  definitions  should  be  established 
early  and  remain  consistent  throughout  the  pro¬ 
gram.  The  program  office  should  assess  the  risks 
of  achieving  performance,  schedule,  and  cost  in 
clear  and  accurate  terms  of  both  probability  and 
consequence.  Where  there  is  disagreement  about 
the  risk,  assessment  efforts  should  be  immediately 
increased.  Confusion  over  risk  is  the  worst  pro¬ 
gram  risk,  because  it  puts  in  doubt  the  validity  of 
the  risk  management  process,  and  therefore, 
whether  program  reality  is  truly  understood. 

The  system  level  risk  assessment  requires  integra¬ 
tion  and  interpretation  of  the  quantified  risk 
assessment  of  the  parts.  This  requires  reasonable 
judgment.  Because  integration  increases  the  po¬ 
tential  for  risk,  it  is  reasonable  to  assume  overall 
risk  is  not  better  than  the  sum  of  objective  data  for 
the  parts. 

Reality  vs.  Expectations 

PMs  are  burdened  with  the  expectations  of 
superiors  and  others  that  have  control  over  the 
program  office’s  environment.  Pressure  to  accom¬ 
modate  these  expectations  is  high.  If  the  systems 
engineer  cannot  communicate  the  reality  of  risk 
in  terms  understandable,  acceptable,  or  sufficiently 
verifiable  to  management,  then  these  pressures  may 
override  vertical  communication  of  actual  risk. 

Formal  systems  engineering  with  risk  management 
incorporated  can  provide  the  verifiable  informa¬ 
tion.  However,  the  systems  engineer  also  has  the 
responsibility  to  adequately  explain  probability  and 
consequences  such  that  the  PM  can  accept  the 
reality  of  the  risk  and  override  higher  level 
expectations. 

Uncertainty  is  a  special  case,  and  very  dangerous 
in  an  atmosphere  of  high  level  expectations. 
Presentation  of  uncertainty  issues  should  strongly 
emphasize  consequences,  show  probability 
trends,  and  develop  “most  likely”  alternatives  for 
probability. 
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SUPPLEMENT  B 

MODEL  FOR  SYSTEM  LEVEL 
RISK  ASSESSMENT 


Because  of  its  rudimentary  nature,  the  model 
presented  here  is  not  meant  to  be  used  as  a  tech¬ 
nique,  but  as  an  illustrative  teaching  tool  to  es¬ 
tablish  the  basic  scope  of  the  three  common  ge¬ 
neric  risk  classifications.  Classifications  should 
be  prepared  by  the  program  office  based  on  the 
risk  assessment  effort.  None  the  less,  the  classi¬ 
fication  definitions  should  be  within  the  general 
scope  of  the  notional  classification  presented 
below. 

Low  Risk  Level  Classifiers 

Risk  Consequences:  Insignificant  cost,  schedule, 
or  technical  impact. 

Probability  of  Occurrence :  Estimated  Probabil¬ 
ity  is  sufficiently  low  to  cause  no  concern. 

Extent  of  Demonstration:  Full  scale  integrated 
technology  has  been  demonstrated  previously. 

Existence  of  Capability:  Already  exists  in  exist¬ 
ing  items  but  needs  to  be  integrated  into  system 
during  development. 

Moderate  Risk  Level  Classifiers 

Risk  Consequences:  Would  affect  program  objec¬ 
tives,  cost  or  schedule.  Cost,  schedule,  and 
performance  are  achievable. 

Probability  of  Occurrence:  Estimate  of  Probabil¬ 
ity  is  high  enough  to  be  of  concern. 


Existence  of  Capability:  Could  exist  in  existing 
items  in  use  today  but  not  at  performance 
standards  for  the  system. 

Extent  of  Changes  Required:  Design  iterations 
necessary. 

High  Risk  Level  Classifiers 

Risk  Consequences:  Significant  program  impact. 
Uncertainties  require  significant  reserve 
requirements  or  alternative  courses  of  action 
and/or  parallel  development. 

Probability  of  Occurrence:  High  estimate  of 
Probability. 

Extent  of  Demonstration:  Technology  has  not 
been  demonstrated. 

Existence  of  Capability:  Does  not  currently  exist. 

Extent  of  Changes  Required:  Significant  design 
iterations  expected  in  order  to  achieve  required/ 
desired  results. 

Intermediate  Classifiers 

Classification  definitions  for  “Low-Moderate”  and 
“Moderate-High”  should  be  developed  if  they  are 
used.  They  should  not  be  left  undefined  and  open 
to  interpretation.  Undefined  classifiers  introduce 
risk  that  is  seldom  recognized,  and  they  tend  to 
result  in  disruptive  consequences. 
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SYSTEMS  ENGINEERING 
PLANNING 


16.1  WHY  ENGINEERING  PLANS? 

Systems  engineering  planning  is  an  activity  that 
has  direct  impact  on  acquisition  planning  decisions 
and  establishes  the  feasible  methods  to  achieve  the 
acquisition  objectives.  Management  uses  it  to: 

•  Assure  that:  all  technical  activities  are  identified 
and  managed, 

•  Communicate  the  technical  approach  to  the 
broad  development  team, 

•  Document  decisions  and  technical  implemen¬ 
tation,  and 

•  Establish  the  criteria  to  judge  how  well  the 
system  development  effort  is  meeting  customer 
and  management  needs. 

Systems  engineering  planning  addresses  the  scope 
of  the  technical  effort  required  to  develop  the 
system.  The  basic  questions  of  “who  will  do  what” 
and  “when”  are  addressed.  As  a  minimum,  a  tech¬ 
nical  plan  describes  what  must  be  accomplished, 
how  systems  engineering  will  be  done,  how  the 
effort  will  be  scheduled,  what  resources  are  needed, 
and  how  the  systems  engineering  effort  will  be 
monitored  and  controlled.  The  planning  effort 
results  in  a  management-oriented  document 
covering  the  implementation  of  program  require¬ 
ments  for  system  engineering,  including  techni¬ 
cal  management  approaches  for  subsequent  phases 
of  the  life  cycle.  In  DoD  it  is  an  exercise  done  on 
a  systems  level  by  the  government,  and  on  a  more 
detailed  level  by  contractors. 


Technical/Systems  Engineering  Planning 

Technical  planning  may  be  documented  in  a  sepa¬ 
rate  engineering  management  plan  or  incorporated 
into  a  broad,  integrated  program  management  plan. 
This  plan  is  first  drafted  at  project  or  program 
inception  during  the  early  requirements  analysis 
effort.  Requirements  analysis  and  technical  plan¬ 
ning  are  inherently  linked,  because  requirements 
analysis  establishes  an  understanding  of  what  must 
be  provided.  This  understanding  is  fundamental 
to  the  development  of  detailed  plans. 

To  be  of  utility,  systems  engineering  plans  must 
be  regularly  updated.  To  support  management 
decision  making,  major  updates  will  usually  occur 
at  least  just  before  major  management  milestone 
decisions.  However,  updates  must  be  performed 
as  necessary  between  management  milestones  to 
keep  the  plan  sufficiently  current  to  achieve  its 
purpose  of  information,  communication,  and 
documentation. 

16.2  ELEMENTS  OF  TECHNICAL  PLANS 

Technical  plans  should  include  sufficient  informa¬ 
tion  to  document  the  purpose  and  method  of  the 
systems  engineering  effort.  Plans  should  include 
the  following: 

•  An  introduction  that  states  the  purpose  of  the 
engineering  effort  and  a  description  of  the 
system  being  developed, 

•  A  technical  strategy  description  that  ties  the 
engineering  effort  to  the  higher-level  manage¬ 
ment  planning, 
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•  A  description  of  how  the  systems  engineering 
process  will  be  tailored  and  structured  to 
complete  the  objectives  stated  in  the  strategy, 

•  An  organization  plan  that  describes  the 
organizational  structure  that  will  achieve  the 
engineering  objectives,  and 

•  A  resource  plan  that  identifies  the  estimated 
funding  and  schedule  necessary  to  achieve  the 
strategy. 

16.21  Introduction 

The  introduction  should  include: 

Scope:  The  scope  of  the  plan  should  provide 
information  concerning  what  part  of  the  big  pic¬ 
ture  the  plan  covers.  For  example,  if  the  plan  were 
a  DOD  program  office  plan,  it  would  emphasize 
control  of  the  higher-level  requirements,  the  system 
definition  (functional  baseline),  and  all  activities 
necessary  for  system  development.  On  the  other 
hand,  a  contractor’s  plan  would  emphasize  control 
of  lower-level  requirements,  preliminary  and 
detail  designs  (allocated  and  product  baselines), 
and  activities  required  and  limited  by  the  con¬ 
tractual  agreement. 

Description:  The  description  of  the  system  should: 

•  Be  limited  to  an  executive  summary  describ¬ 
ing  those  features  that  make  the  system  unique, 

•  Include  a  general  discussion  of  the  system’s 
operational  functions,  and 

•  Answer  the  question  “What  is  it  and  what  will 
it  do?” 

Focus:  A  guiding  focus  for  the  effort  should  be 
provided  to  clarify  the  management  vision  for  the 
development  approach.  For  example,  the  focus 
may  be  lowest  cost  to  obtain  threshold  require¬ 
ments,  superior  performance  within  budget, 
superior  standardization  for  reduced  logistics, 
maximum  use  of  the  open  systems  approach  to  re¬ 
duce  cost,  or  the  like.  A  focus  statement  should: 


•  Be  a  single  objective  to  avoid  confusion, 

•  Be  stated  simply  to  avoid  misinterpretation,  and 

•  Have  high-level  support. 

Purpose:  The  purpose  of  the  engineering  effort 
should  be  described  in  general  terms  of  the  outputs, 
both  end  products  and  life-cycle  enabling  prod¬ 
ucts  that  are  required.  The  stated  purpose  should 
answer  the  question,  “What  does  the  engineer¬ 
ing  effort  have  to  produce?” 

16.22  Technical  Strategy 

The  basic  purpose  of  a  technical  strategy  is  to  link 
the  development  process  with  the  acquisition  or 
contract  management  process.  It  should  include: 

•  Development  phasing  and  associated  baselining, 

•  Key  engineering  milestones  to  support  risk  man¬ 
agement  and  business  management  milestones, 

•  Associated  parallel  developments  or  product 
improvement  considerations,  and 

•  Other  management  generated  constraints  or 
high-visibility  activities  that  could  affect  the 
engineering  development. 

Phasing  and  Milestones:  The  development 
phasing  and  baseline  section  should  describe  the 
approach  to  phasing  the  engineering  effort, 
including  tailoring  of  the  basic  process  described 
in  this  book  and  a  rationale  for  the  tailoring.  The 
key  milestones  should  be  in  general  keeping  with 
the  technical  review  process,  but  tailored  as 
appropriate  to  support  business  management 
milestones  and  the  project/program’s  development 
phasing.  Strategy  considerations  should  also 
include  discussion  of  how  design  and  verification 
will  phase  into  production  and  fielding.  This  area 
should  identify  how  production  will  be  phased-in 
(including  use  of  limited-rate  initial  production  and 
long  lead-time  purchases),  and  that  initial  support 
considerations  require  significant  coordination 
between  the  user  and  acquisition  community. 
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Parallel  Developments  and  Product  Improve¬ 
ment:  Parallel  development  programs  necessary 
for  the  system  to  achieve  its  objectives  should  be 
identified  and  the  relationship  between  the  efforts 
explained.  Any  product  improvement  strategies 
should  also  be  identified.  Considerations  such  as 
evolutionary  development  and  pre-planned  prod¬ 
uct  improvement  should  be  described  in  sufficient 
detail  to  show  how  they  would  phase  into  the  over¬ 
all  effort. 

16.23  Impacts  on  Strategy 

All  conditions  or  constraints  that  impact  the  strat¬ 
egy  should  be  identified  and  the  impact  assessed. 
Key  points  to  consider  are: 

•  Critical  technologies  development, 

•  Cost  As  an  Independent  Variable  (CAIV),  and 

•  Any  business  management  directed  constraint 
or  activity  that  will  have  a  significant  influence 
on  the  strategy. 

Critical  Technologies:  Discussion  of  critical 
technology  should  include: 

•  Risk  associated  with  critical  technology 
development  and  its  impact  on  the  strategy, 

•  Relationship  to  baseline  development,  and 

•  Potential  impact  on  the  overall  development 
effort. 

CAIV:  Strategy  considerations  should  include 
discussion  of  how  Cost  As  an  Independent  Variable 
(CAIV)  will  be  implemented,  and  how  it  will 
impact  the  strategy.  It  should  discuss  how  unit  cost, 
development  cost,  life  cycle  cost,  total  ownership 
cost,  and  their  interrelationships  apply  to  the  sys¬ 
tem  development.  This  area  should  focus  on  how 
these  costs  will  be  balanced,  how  they  will  be  con¬ 
trolled,  and  what  impact  they  have  on  the  strategy 
and  design  approach. 

Management  Issues:  Management  issues  that 
pose  special  concerns  for  the  development  strategy 


could  cover  a  wide  range  of  possible  issues.  In 
general,  management  issues  identified  as  engineer¬ 
ing  strategy  issues  are  those  that  impact  the  abil¬ 
ity  to  support  the  management  strategy.  Examples 
would  include: 

•  Need  to  combine  developmental  phases  to 
accommodate  management  driven  schedule  or 
resource  limitations, 

•  Risk  associated  with  a  tight  schedule  or  limited 
budget, 

•  Contractual  approach  that  increases  technical 
risk,  and 

•  Others  of  a  similar  nature. 

Management-dictated  technical  activities — such  as 
use  of  modeling  and  simulation,  open  systems, 
IPPD,  and  others — should  not  be  included  as  a 
strategy  issue  unless  they  impact  the  overall  sys¬ 
tems  engineering  strategy  to  meet  management 
expectations.  The  strategy  discussion  should  lay 
out  the  plan,  how  it  dovetails  with  the  manage¬ 
ment  strategy,  and  how  management  directives 
impact  it. 

16.24  Systems  Engineering  Processes 

This  area  of  the  planning  should  focus  on  how  the 
system  engineering  processes  will  be  designed  to 
support  the  strategy.  It  should  include: 

•  Specific  methods  and  techniques  used  to 
perform  the  steps  and  loops  of  the  systems 
engineering  process, 

•  Specific  system  analysis  and  control  tools  and 
how  they  will  be  used  to  support  step  and  loop 
activities,  and 

•  Special  design  considerations  that  must  be 
integrated  into  the  engineering  effort. 

Steps  and  Loops:  The  discussion  of  how  the 
systems  engineering  process  will  be  done  should 
show  the  specific  procedures  and  products  that  will 
ensure: 
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•  Requirements  are  understood  prior  to  the 
flow-  down  and  allocation  of  requirements, 

•  Functional  descriptions  are  established  before 
designs  are  formulated, 

•  Designs  are  formulated  that  are  traceable  to 
requirements, 

•  Methods  exist  to  reconsider  previous  steps,  and 

•  Verification  processes  are  in  place  to  ensure  that 
design  solutions  meet  needs  and  requirements. 

This  planning  area  should  address  each  step  and 
loop  for  each  development  phase,  include  identi¬ 
fication  of  the  step  specific  tools  (Functional  Flow 
Block  Diagrams,  Timeline  Analysis,  etc.)  that  will 
be  used,  and  establish  the  verification  approach. 
The  verification  discussion  should  identify  all 
verification  activities,  the  relationship  to  formal 
developmental  T&E  activities,  and  independent 
testing  activities  (such  as  operational  testing). 

Norms  of  the  particular  technical  area  and  the 
engineering  processes  of  the  command,  agency, 
or  company  doing  the  tasks  will  greatly  influence 
this  area  of  planning.  However,  whatever  proce¬ 
dures,  techniques,  and  analysis  products  or  models 
used,  they  should  be  compatible  with  the  basic 
principles  of  systems  engineering  management  as 
described  earlier  in  this  book. 

An  example  of  the  type  of  issue  this  area  would 
address  is  the  requirements  analysis  during  the 
system  definition  phase.  Requirements  analysis  is 
more  critical  and  a  more  central  focus  during  sys¬ 
tem  definition  than  in  later  phases.  The  establish¬ 
ment  of  the  correct  set  of  customer  requirements 
at  the  beginning  of  the  development  effort  is 
essential  to  proper  development.  Accordingly,  the 
system  definition  phase  requirements  analysis 
demands  tight  control  and  an  early  review  to  verify 
the  requirements  are  established  well  enough  to 
begin  the  design  effort.  This  process  of  control 
and  verification  necessary  for  the  system  defini¬ 
tion  phase  should  be  specifically  described  as  part 
of  the  overall  requirements  analysis  process  and 
procedures. 


Analysis  and  Control:  Planning  should  identify 
those  analysis  tools  that  will  be  used  to  evaluate 
alternative  approaches,  analyze  or  assess  effective¬ 
ness,  and  provide  a  rigorous  quantitative  basis  for 
selecting  performance,  functional,  and  design 
requirements.  These  processes  can  include  trade 
studies,  market  surveys,  modeling  and  simulation, 
effectiveness  analyses,  design  analyses,  QFD, 
design  of  experiments,  and  others. 

Planning  must  identify  the  method  by  which 
control  and  feedback  will  be  established  and 
maintained.  The  key  to  control  is  performance- 
based  measurement  guided  by  an  event-based 
schedule.  Entrance  and  exit  criteria  for  the  event- 
driven  milestones  should  be  established  sufficient 
to  demonstrate  proper  development  progress  has 
been  completed.  Event-based  schedules  and  exit 
criteria  are  further  discussed  later  in  this  chapter. 
Methods  to  maintain  feedback  and  control  are 
developed  to  monitor  progress  toward  meeting  the 
exit  criteria.  Common  methods  were  discussed 
earlier  in  this  book  in  the  chapters  on  metrics,  risk 
management,  configuration  management,  and 
technical  reviews. 

Design  Considerations:  In  every  system  devel¬ 
opment  there  are  usually  technical  activities  that 
require  special  attention.  These  may  come  from 
management  concerns,  legal  or  regulatory  direc¬ 
tives,  social  issues,  or  organizational  initiatives. 
For  example,  a  DoD  program  office  will  have  to 
conform  to  DoDD  5000.2-R,  which  lists  several 
technical  activities  that  must  be  incorporated  into 
the  development  effort.  DoD  plans  should  specifi¬ 
cally  address  each  issue  presented  in  Part  4  of  DoD 
5000.2-R. 

In  the  case  of  a  contractor  there  may  be  issues 
delineated  in  the  contract,  promised  in  the  pro¬ 
posal,  or  established  by  management  that  the  tech¬ 
nical  effort  must  address.  The  system  engineering 
planning  must  describe  how  each  of  these  issues 
will  be  integrated  into  the  development  effort. 

16.25  Organization 

Systems  engineering  management  planning  should 
identify  the  basic  structure  that  will  develop  the 
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system.  Organizational  planning  should  address 
how  the  integration  of  the  different  technical  dis¬ 
ciplines,  primary  function  managers,  and  other 
stakeholders  will  be  achieved  to  develop  the 
system.  This  planning  area  should  describe  how 
multi-disciplinary  teaming  would  be  implemented, 
that  is,  how  the  teams  will  be  organized,  tasked, 
and  trained.  A  systems-level  team  should  be 
established  early  to  support  this  effort.  Roles, 
authority,  and  basic  responsibilities  of  the  system- 
level  design  team  should  be  specifically  described. 
Establishing  the  design  organization  should  be  one 
of  the  initial  tasks  of  the  system-level  design  team. 
Their  basic  approach  to  organizing  the  effort  should 
be  described  in  the  plan.  Further  information  on 
organizing  is  contained  in  a  later  chapter. 

16.26  Resources 

The  plan  should  identify  the  budget  for  the  tech¬ 
nical  development.  The  funds  required  should  be 
matrixed  against  a  calendar  schedule  based  on  the 
event-based  schedule  and  the  strategy.  This  should 
establish  the  basic  development  timeline  with  an 
associated  high-level  estimated  spending  profile. 
Short  falls  in  funding  or  schedule  should  be 
addressed  and  resolved  by  increasing  funds, 
extending  schedule,  or  reducing  requirements  prior 
to  the  plan  preparation.  Remember  that  future 
analysis  of  development  progress  by  management 
will  tend  to  be  based  on  this  budget  “promised”  at 
plan  inception. 

16.3  INTEGRATION  OF  PLANS  - 
PROGRAM  PLAN  INTERFACES 

Systems  engineering  management  planning  must 
be  coordinated  with  interfacing  activities  such 
as  these: 

•  Acquisition  Strategy  assures  that  technical 
plans  take  into  account  decisions  reflected  in 
the  Acquisition  Strategy.  Conflicts  must  be 
identified  early  and  resolved. 

•  Financial  plan  assures  resources  match  the 
needs  in  the  tech  plan.  Conflicts  should  be 
identified  early  and  resolved. 


•  Test  and  Evaluation  Master  Plan  (TEMP) 
assures  it  complements  the  verification  ap¬ 
proach.  It  should  provide  an  integrated  ap¬ 
proach  to  verify  that  the  design  configuration 
will  meet  customer  requirements.  This  ap¬ 
proach  should  be  compatible  with  the  verifi¬ 
cation  approach  delineated  in  the  systems  en¬ 
gineering  plan. 

•  Configuration  management  plan  assures  that 
the  development  process  will  maintain  the 
system  baselines  and  control  changes  to  them. 

•  Design  plans  (e.g.,  electrical,  mechanical,  struc¬ 
tural,  etc)  coordinates  identification  of  IPT 
team  composition. 

•  Integrated  logistics  support  planning  and 
support  analysis  coordinates  total  system 
support. 

•  Production/Manufacturing  plan  to  coordinate 
activities  concerning  design  producibility,  and 
follow-on  production, 

•  Quality  management  planning  assures  that 
quality  engineering  activities  and  quality  man¬ 
agement  functions  are  included  in  system 
engineering  planning, 

•  Risk  management  planning  establishes  and 
coordinates  technical  risk  management  to 
support  total  program  risk  management. 

•  Interoperability  planning  assures  interoper¬ 
ability  suitability  issues  are  coordinated  with 
system  engineering  planning.  (Where  interop¬ 
erability  is  an  especially  critical  requirement 
such  as,  communication  or  information  sys¬ 
tems,  it  should  be  addressed  as  a  separate  issue 
with  separate  integrated  teams,  monitoring, 
and  controls). 

•  Others  such  as  modeling  and  simulation  plan, 
software  development  plan,  human  integration 
plan,  environment,  safety  and  health  planning, 
also  interface. 


137 


Systems  Engineering  Fundamentals 


Chapter  16 


Things  to  Watch 

A  well  developed  technical  management  plan  will 

include: 

•  The  expected  benefit  to  the  user, 

•  How  a  total  systems  development  will  be 
achieved  using  a  systems  engineering  approach, 

•  How  the  technical  plan  complements  and  sup¬ 
ports  the  acquisition  or  management  business 
plan, 

•  How  incremental  reviews  will  assure  that  the 
development  stays  on  track, 

•  How  costs  will  be  reduced  and  controlled, 

•  What  technical  activities  are  required  and  who 
will  perform  them, 

•  How  the  technical  activities  relate  to  work 
accomplishment  and  calendar  dates, 

•  How  system  configuration  and  risk  will  be 
controlled, 

•  How  system  integration  will  be  achieved, 


•  How  the  concerns  of  the  eight  primary  life 
cycle  functions  will  be  satisfied, 

•  How  regulatory  and  contractual  requirements 
will  be  achieved,  and 

•  The  feasibility  of  the  plan,  i.e.,  is  the  plan 
practical  and  executable  from  a  technical, 
schedule,  and  cost  perspective. 

16.4  SUMMARY  POINTS 

•  Systems  engineering  planning  should  establish 
the  organizational  structure  that  will  achieve 
the  engineering  objectives. 

•  Planning  must  include  event-based  scheduling 
and  establish  feedback  and  control  methods. 

•  It  should  result  in  important  planning  and 
control  documents  for  carrying  out  the 
engineering  effort. 

•  It  should  identify  the  estimated  funding  and 
detail  schedule  necessary  to  achieve  the  strategy. 

•  Systems  engineering  planning  should  establish 
the  proper  relationship  between  the  acquisition 
and  technical  processes. 
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APPENDIX  A 

SCHEDULES 


The  event-based  schedule,  sometimes  referred 
to  as  the  Systems  Engineering  Master  Schedule 
(SEMS)  or  Integrated  Master  Schedule  (IMS)  is 
a  technical  event  driven  (not  time  driven)  plan 
primarily  concerned  with  product  and  process 
development.  It  forms  the  basis  for  schedule  con¬ 
trol  and  progress  measurement,  and  relates 
engineering  management  events  and  accomplish¬ 
ments  to  the  WBS.  These  events  are  identified 
either  in  the  format  of  entry  and  exit  events  (e.g. 
initiate  PDR,  complete  PDR)  or  by  using  entry 
and  exit  criteria  for  each  event.  Example  exit 
criteria  shown  in  Figures  16-1  and  16-2. 


The  program  office  develops  an  event-based 
schedule  that  represents  the  overall  development 
effort.  This  schedule  is  usually  high-level  and 
focused  on  the  completion  of  events  that  support 
the  acquisition  milestone  decision  process.  An 
event-based  schedule  is  developed  by  the  contrac¬ 
tor  to  include  significant  accomplishments  that 
must  be  completed  in  order  to  meet  the  progress 
required  prior  to  contract  established  events.  The 
contractor  also  includes  events,  accomplishments, 
and  associated  success  criteria  specifically  identi¬ 
fied  by  the  contract.  DoD  program  offices  can  use 


System  Requirements 
Review  (SRR) 

System  Functional 
Review/Software  Spec 
Review(SFR/SSR) 

Preliminary  Design 
Review  (PDR) 

Critical  Des  Review 

Test  Readiness  Review 
(CDR/TRR) 

•  Mission  Analysis 
completed 

•  Support  Strategy  defined 

•  System  options  decisions 
completed 

•  Design  usage  defined 

•  Op  performance  reqmt 
defined 

•  Manpower  sensitivities 
completed 

•  Operational  architecture 
available  and  reviewed 

•  installed  environments 
defined 

•  Maintenance  concept 
defined 

•  Preliminary  design  criteria 
established 

•  Preliminary  design  margins 
established 

•  Interfaces  defined/ 
preliminary  interface  specs 
completed 

•  Software  and  software 
support  requirements 
completed 

•  Baseline  support/resources 
requirements  defined 

•  Support  equipment 
capability  defined 

•  Technical  architecture 
prepared 

•  System  defined  and 
requirements  shown  to  be 
achievable 

•  Design  analyses/definition 
completed 

•  Material/parts  characteriza¬ 
tion  completed 

•  Design  maintainability 
analysis  completed/support 
requirements  defined 

•  Preliminary  production  plan 
completed 

•  Make/buy  decisions 
finalized 

•  Breadboard  investigations 
completed 

•  Coupon  testing  completed 

•  Design  margins  completed 

•  Preliminary  FMECA 
completed 

•  Software  functions  and 
architecture  and  support 
defined 

•  Maintenance  tasks  trade 
studies  completed 

•  Support  equipment 
development  specs 
completed 

•  Parts,  materials,  processes 
selected 

•  Development  tests 
completed 

•  Inspection  points/criteria 
completed 

•  Component  level  FMECA 
completed 

•  Repair  level  analysis 
completed 

•  Facility  requirements 
defined 

•  Software  test  descriptions 
completed 

•  Hardware  and  software 
hazard  analysis  completed 

•  Firmware  spt  completed 

•  Software  programmers 
maual  completed 

•  Durability  test  completed 

•  Maintinability  analyses 
completed 

•  Qualification  test  proce¬ 
dures  approved 

•  Producibility  analyses 
completed 

Figure  16-1.  Example  Event-Based  Schedule  Exit  Criteria 
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System  Verfication  Review/ 

Functional  Configuration  Audit 
(SVR/FCA) 

Physical  Configuration  Audit 
(PCA) 

•  All  verification  tasks  completed 

•  Durability  tests  completed 

•  Long  lead  time  items  identified 

•  PME  and  operational  training  completed 

•  Tech  manuals  completed 

•  Flight  test  plan  approved 

•  Support  and  training  equipment  developed 

•  Fielding  analysis  completed 

•  Provisioning  data  verified 

•  Qualification  testing  completed 

•  All  QA  provisions  finalized 

•  All  manufacturing  process  requirements  and 
documentation  finalized 

•  Product  fabrication  specifications  finalized 

•  Support  and  training  equipment  qualification 
completed 

•  All  acceptance  test  requirements  completed 

•  Life  management  plan  completed 

•  System  support  capability  demonstrated 

•  Post  production  support  analysis  completed 

•  Final  software  description  document  and  all 
user  manuals  complete 

Figure  16-2.  Example  Event-Driven  Schedule  Exit  Criteria  (continued) 


the  contractor's  event-based  schedule  and  the 
contractor's  conformance  to  it  for  several  pur¬ 
poses:  source  selection,  monitoring  contractor 
progress,  technical  and  other  reviews,  readiness 
for  option  award,  incentives/awards  determina¬ 
tion,  progress  payments  decision,  and  similar 
activities. 

The  event-based  schedule  establishes  the  key  pa¬ 
rameters  for  determining  the  progress  of  a  devel¬ 
opment  program.  To  some  extent  it  controls  and 
interfaces  with  systems  engineering  management 
planning,  integrated  master  schedules  and  inte¬ 
grated  master  plans,  as  well  as  risk  management 
planning,  system  test  planning,  and  other  key  plans 
which  govern  the  details  of  program  management. 

The  calendar  or  detail  schedule  is  a  time-based 
schedule  that  shows  how  work  efforts  will  sup¬ 
port  tasks  and  events  identified  in  the  event-based 


schedule.  It  aligns  the  tasks  and  calendar  dates 
to  show  when  each  significant  accomplishment 
must  be  achieved.  It  is  a  key  component  for  de¬ 
veloping  Earned  Value  metrics.  The  calendar 
schedule  is  commonly  referred  to  as  the  detail 
schedule,  systems  engineering  detail  schedule, 
or  SEDS.  The  contractor  is  usually  required  to 
maintain  the  relationship  between  the  event  and 
calendar  schedules  for  contract  required  activi¬ 
ties.  Figure  16-3  shows  the  relationship  between 
the  system  requirements,  the  WBS,  the  contrac¬ 
tual  requirements,  the  event-based  schedule,  and 
the  detail  schedule. 

Schedule  Summary 

The  event-based  schedule  establishes  the  key  tasks 
and  results  expected.  The  event-based  schedule 
establishes  the  basis  for  a  valid  calendar-based 
(detail)  schedule. 
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Significant  Accomplishments 


1.  Preliminary  Design  Complete  X 


1.  a.  Duty  Cycle  Defined 

b.  Preliminary  Analysis  Complete/Reviewed 

c.  Preliminary  Drawings  Released 


Detailed  Tasks 

20XX 

20XY 

20XZ 

Program  Events: 

pdrA 

cdrA 

1.  Preliminary  Design  Complete 

Duty  Cycle  Defined 

▲  . _A 

Figure  16-3.  Event-Based  Detailed  Schedule  Interrelationships 
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17.1  INTRODUCTION 

Complex  systems  do  not  usually  have  stagnant 
configurations.  A  need  for  a  change  during  a 
system’s  life  cycle  can  come  from  many  sources 
and  effect  the  configuration  in  infinite  ways.  The 
problem  with  these  changes  is  that,  in  most  cases 
it  is  difficult,  if  not  impossible,  to  predict  the  nature 
and  timing  of  these  changes  at  the  beginning  of 
system  development.  Accordingly,  strategies  or 
design  approaches  have  been  developed  to  reduce 
the  risk  associated  with  predicted  and  unknown 
changes. 

Well  thought-out  improvement  strategies  can  help 
control  difficult  engineering  problems  related  to: 

•  Requirements  that  are  not  completely  under¬ 
stood  at  program  start, 

•  Technology  development  that  will  take  longer 
than  the  majority  of  the  system  development, 

•  Customer  needs  (such  as  the  need  to  combat  a 
new  military  threat)  that  have  increased,  been 
upgraded,  are  different,  or  are  in  flux, 

•  Requirements  change  due  to  modified  policy, 
operational  philosophy,  logistics  support 
philosophy,  or  other  planning  or  practices  from 
the  eight  primary  life  cycle  function  groups, 

•  Technology  availability  that  allows  the  system 
to  perform  better  and/or  less  expensively, 

•  Potential  reliability  and  maintainability  up¬ 
grades  that  make  it  less  expensive  to  use, 
maintain,  or  support,  including  development  of 
new  supply  support  sources, 


•  Safety  issues  requiring  replacement  of  unsafe 
components,  and 

•  Service  life  extension  programs  that  refurbish 
and  upgrade  systems  to  increase  their  service  life. 

In  DoD,  the  21st  century  challenge  will  be 
improving  existing  products  and  designing  new 
ones  that  can  be  easily  improved.  With  the  aver¬ 
age  service  life  of  a  weapons  system  in  the  area  of 
40  or  more  years,  it  is  necessary  that  systems  be 
developed  with  an  appreciation  for  future  require¬ 
ments,  foreseen  and  unforeseen.  These  future 
requirements  will  present  themselves  as  needed 
upgrades  to  safety,  performance,  supportability, 
interface  compatibility,  or  interoperability;  changes 
to  reduce  cost  of  ownership;  or  major  rebuild. 
Providing  these  needed  improvements  or  correc¬ 
tions  form  the  majority  of  the  systems  engineer’s 
post-production  activities. 

17.2  PRODUCT  IMPROVEMENT 
STRATEGIES 

As  shown  by  Figure  17-1,  these  strategies  vary 
based  on  where  in  the  life  cycle  they  are  applied. 
The  strategies  or  design  approaches  that  reflect 
these  improvement  needs  can  be  categorized  as 
planned  improvements,  changes  in  design  or 
production,  and  deployed  system  upgrades. 

Planned  Improvements 

Planned  improvements  strategies  include  evolu¬ 
tionary  acquisition,  pre-planned  product  develop¬ 
ment,  and  open  systems.  These  strategies  are  not 
exclusive  and  can  be  combined  synergistically  in 
a  program  development. 
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Requirements  Analysis 
•  General  for  the  System 
•  Specific  for  the  Core 


Concept  of  Operations 


Preliminary 

System 

Architecture 


' - ►Define  -  FUND  -  Develop  -  Operationally  Test  CORE 


CORE 

BlockX  ''N 

Requirements 


Define  -  FUND  -  Develop  -  Operationally  Test  Block  A 


Customer 
Feedback 
“Managed” 
by  Req 
Analysis 


The  lack  of  specificity 
and  detail  in  identifying  the  final 
system  capability  is  what 
distinguishes  Evolutionary 
Acquisition  from  an 
acquisition  strategy  based 


1 _ continue  as  require 

°  . |  -  JLC  EA  Guide  f 

Flexible/Incremental  ORD,  TEMP,  etc. 

L-+  I 

Figure  17-2.  Evolutionary  Acquisition 
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Evolutionary  Acquisition:  Evolutionary  acquisi¬ 
tion  is  an  appropriate  strategy  where  a  core 
requirement  can  be  identified,  but  requirements 
or  technology  growth  is  expected.  Figure  17-2 
illustrates  the  concept.  The  core  is  designed  and 
built,  but  follow-on  versions  of  the  system  include 
block  upgrades  as  more  is  learned  about  require¬ 
ments  and  technology.  Two  key  characteristics  of 
this  approach  are  requirements  flexibility  and  bud¬ 
get  constraints.  Requirements  are  refined  periodi¬ 
cally  in  response  to  technology,  user  feedback,  and 
budget  opportunities.  These  programs  require  con¬ 
tinual  study  of  ways  to  increase  or  optimize  the 
system  capability  within  budget,  and  continual 
prioritization  of  possible  upgrades.  The  decision 
of  what  upgrades  are  appropriate  is  combined  with 
the  decision  of  what  annual  budget  is  available  or 
obtainable.  Evolutionary  acquisition  requires  plan¬ 
ning  for  incremental  upgrades  throughout  design 
and  production.  To  achieve  this  the  system  is 
designed  for  change  flexibility  using  techniques 
such  as  open  systems,  modular  designs,  component 
replacement,  and  the  like. 


Preplanned  Product  Improvement:  Often  referred 
to  as  P3I,  preplanned  product  improvement  is  an 
appropriate  strategy  when  requirements  are  known 
and  firm,  but  where  constraints  (typically  either 
technology  or  budget)  make  some  portion  of  the 
system  unachievable  within  the  schedule  required. 
If  it  is  concluded  that  a  militarily  useful  capability 
can  be  fielded  as  an  interim  solution  while  the 
portion  yet  to  be  proceeds  through  development, 
then  P3I  is  appropriate.  The  approach  generally  is 
to  handle  the  improvement  as  a  separate,  parallel 
development;  initially  test  and  deliver  the  system 
without  the  improvement;  and  prove  and  provide 
the  enhanced  capability  as  it  becomes  available. 
The  key  to  a  successful  pre-planned  product 
improvement  is  the  establishment  of  well-defined 
interface  requirements  for  the  system  and  the  improv¬ 
ement.  Use  of  a  pre-planned  product  improvement 
will  tend  to  increase  initial  cost,  configuration 
management  activity,  and  technical  complexity. 
Figure  17-3  shows  some  of  the  considerations  in 
deciding  when  it  is  appropriate. 


Responsive  to  threat  changes 
’  Accommodates  future  technology 
'  IOC  can  be  earlier 
'  Reduced  development  risk 
'  Possible  subsystem  competition 
’  increased  effective  operational  life 


Acquisition  Issues 

Longer  Range  Planning 
Parallel  Efforts 

Standards  and  Interface  Capacity 
Modular  Equipment/Open  Systems 


The  P3I  acquisition 
management  challenge  is  to  acquire 
systems  with  interfaces  and  accessibility 
as  an  integral  part  of  the  design  so  that 
e  deferred  elements)  can  be  incorporated 
in  a  cost-effective  manner  when  they 
become  available. 


•  Increased  initial  development  cost 

•  Increased  technical  requirements 
complexity 

•  More  complex  CM 

•  Sensitive  to  funding  streams 

•  Parallel  development  management 


Figure  17-3.  Pre-Planned  Product  Improvement 
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Open  Systems  Approach:  The  open  system 
design  approach  uses  interface  management  to 
build  flexible  design  interfaces  that  accommo¬ 
date  use  of  competitive  commercial  products  and 
provide  enhanced  capacity  for  future  change.  It 
can  be  used  to  prepare  for  future  needs  when 
technology  is  yet  not  available,  whether  the  op¬ 
erational  need  is  known  or  unknown.  The  open 
systems  focus  is  to  design  the  system  such  that 
it  is  easy  to  modify;  using  standard  interfaces, 
modularity,  recognized  interface  standards,  stan¬ 
dard  components  with  recognized  common  in¬ 
terfaces,  commercial  and  non-developmental 
items,  and  compartmentalized  design.  Open  sys¬ 
tem  approaches  to  design  are  further  discussed 
at  the  end  of  this  chapter. 

Changes  in  Design  or  Production 

Engineering  Change  Proposals  (ECPs): 
Changes  that  are  to  be  implemented  during  the 
development  and  production  of  a  given  system 
are  typically  initiated  through  the  use  of  engi¬ 
neering  change  proposals  (ECPs).  If  the  pro¬ 
posed  change  is  approved  (usually  by  a  con¬ 
figuration  control  board)  the  changes  to  the 
documentation  that  describes  the  system  are 
handled  by  formal  configuration  management, 
since,  by  definition,  ECPs,  when  approved, 
change  an  approved  baseline.  ECPs  govern  the 
scope  and  details  of  these  changes.  ECPs  may 
address  a  variety  of  needs,  including  correc¬ 
tion  of  deficiencies,  cost  reduction,  and  safety. 
Furthermore,  ECPs  may  been  assigned  differ¬ 
ing  levels  of  priority  from  routine  to  emergency. 
MIL-HDBK-61,  Configuration  Management 
Guidance,  offers  an  excellent  source  of  advice 
on  issues  related  to  configuration  changes. 

Block  Change  before  Deployment:  Block  changes 
represent  an  attempt  to  improve  configuration 
management  by  having  a  number  of  changes 
grouped  and  applied  such  that  they  will  apply  con¬ 
sistently  to  groups  (or  blocks)  of  production  items. 
This  improves  the  management  and  configuration 
control  of  similar  items  substantially  in  compari¬ 
son  to  change  that  is  implemented  item  by  item 
and  single  change  order  by  single  change  order. 
When  block  changes  occur,  the  life  cycle  impact 


should  be  carefully  addressed.  Significant  dif¬ 
ferences  in  block  configurations  can  lead  to  dif¬ 
ferent  manuals,  supply  documentation,  training, 
and  restrictions  as  to  locations  or  activities  where 
the  system  can  be  assigned. 

Deployed  Systems  Upgrades 

Major  Rebuild:  A  major  rebuild  results  from  the 
need  for  a  system  that  satisfies  requirements  sig¬ 
nificantly  different  or  increased  from  the  existing 
system,  or  a  need  to  extend  the  life  of  a  system 
that  is  reaching  the  end  of  its  usable  life.  In  both 
cases  the  system  will  have  upgraded  requirements 
and  should  be  treated  as  basically  a  new  system 
development.  A  new  development  process  should 
be  started  to  establish  and  control  configuration 
baselines  for  the  rebuilt  system  based  on  the 
updated  requirements. 

Major  rebuilds  include  re-manufacturing,  service- 
life  extension  programs,  and  system  developments 
where  significant  parts  of  a  previous  system  will 
be  reused.  Though  rebuilding  existing  systems  can 
dramatically  reduce  the  cost  of  a  new  system  in 
some  cases,  the  economies  of  rebuild  can  be 
deceiving,  and  the  choice  of  whether  to  pursue  a 
rebuild  should  be  done  after  careful  use  of  trade 
studies.  The  key  to  engineering  such  systems  is  to 
remember  that  they  are  new  systems  and  require 
the  full  developmental  considerations  of 
baselining,  the  systems  engineering  process,  and 
life  cycle  integration. 

Post-Production  Improvement:  In  general,  prod¬ 
uct  improvements  become  necessary  to  improve 
the  system  or  to  maintain  the  system  as  its  com¬ 
ponents  reach  obsolescence.  These  projects  gen¬ 
erally  result  in  a  capability  improvement,  but 
for  all  practical  purposes  the  system  still  the 
serves  the  same  basic  need.  These  improvements 
are  usually  characterized  by  an  upgrade  to  a  com¬ 
ponent  or  subsystem  as  opposed  to  a  total  sys¬ 
tem  upgrade. 

Block  Upgrades:  Post-production  block  up¬ 
grades  are  improvements  to  a  specific  group  of 
the  system  population  that  provides  a  consistent 
configuration  within  that  group.  Block  upgrades 
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in  post-production  serve  the  same  general  pur¬ 
pose  of  controlling  individual  system  configura¬ 
tions  as  production  block  upgrades,  and  they  re¬ 
quire  the  same  level  of  life  cycle  integration. 

Modifying  an  Existing  System 

Upgrading  an  existing  system  is  a  matter  of  fol¬ 
lowing  the  system  engineering  process,  with  an 
emphasis  on  configuration  and  interface  manage¬ 
ment.  The  following  activities  should  be  included 
when  upgrading  a  system: 

•  Benchmark  the  modified  requirements  both  for 
the  upgrade  and  the  system  as  a  whole, 

•  Perform  functional  analysis  and  allocation  on 
the  modified  requirements, 

•  Assess  the  actual  capability  of  the  pre-upgrade 
system, 

•  Identify  cost  and  risk  factors  and  monitor  them, 

•  Develop  and  evaluate  modified  system 
alternatives, 

•  Prototype  the  chosen  improvement  alternative, 
and 

•  Verify  the  improvement. 

Product  improvement  requires  special  attention  to 
configuration  and  interface  management.  It  is  not 
uncommon  that  the  existing  system’s  configura¬ 
tion  will  not  be  consistent  with  the  existing  con¬ 
figuration  data.  Form,  fit,  and  especially  function 
interfaces  often  represent  design  constraints  that 
are  not  always  readily  apparent  at  the  outset  of  a 
system  upgrade.  Upgrade  planning  should  ensure 
that  the  revised  components  will  be  compatible  at 
the  interfaces.  Where  interfaces  are  impacted, 
broad  coordination  and  agreement  is  normally 
required. 

Traps  in  Upgrading  Deployed  Systems 

When  upgrading  a  deployed  system  pay  attention 
to  the  following  significant  traps: 


Scheduling  to  minimize  operational  impacts:  The 
user’s  operational  commitments  will  dictate  the 
availability  of  the  system  for  modification.  If 
the  schedule  conflicts  with  an  existing  or  emerg¬ 
ing  operational  need,  the  system  will  probably  not 
become  available  for  modification  at  the  time 
agreed  to.  Planning  and  contractual  arrangements 
must  be  flexible  enough  to  accept  unforeseen 
schedule  changes  to  accommodate  user’s  unan¬ 
ticipated  needs. 

Configuration  and  interface  management:  Con¬ 
figuration  management  must  address  three 
configurations:  the  actual  existing  configuration, 
the  modification  configuration,  and  the  final 
system  configuration.  The  key  to  successful  modi¬ 
fication  is  the  level  of  understanding  and  control 
associated  with  the  interfaces. 

Logistics  compatibility  problems:  Modification 
will  change  the  configuration,  which  in  most  cases 
will  change  the  supply  support  and  maintenance 
considerations.  Coordination  with  the  logistics 
community  is  essential  to  the  long-term  operational 
success  of  the  modification. 

Minimal  resources  available:  Modifications  tend 
to  be  viewed  as  simple  changes.  As  this  chapter 
has  pointed  out,  they  are  not;  and  they  should  be 
carefully  planned.  That  planning  should  include 
an  estimate  of  needed  resources.  If  the  resources 
are  not  available,  either  the  project  should  be 
abandoned,  or  a  plan  formulated  to  mitigate  and 
control  the  risk  of  an  initial,  minimal  budget 
combined  with  a  plan  for  obtaining  additional 
resources. 

Limited  competitors:  Older  systems  may  have  only 
a  few  suppliers  that  have  a  corporate  knowledge 
of  the  particular  system  functions  and  design.  This 
is  especially  problematic  if  the  original  system 
components  were  commercial  or  non-developmen- 
tal  items  that  the  designer  does  not  have  product 
baseline  data  for.  In  cases  such  as  these,  there  is  a 
learning  process  that  must  take  place  before  the 
designer  or  vendor  can  adequately  support  the 
modification  effort.  Depending  on  the  specific 
system,  this  could  be  a  major  effort.  This  issue 
should  be  considered  very  early  in  the 
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Funding  restrictions  ($  color)  drive  the  need  to  separate 
performance  increase  from  supportability  changes 


Product  improvement  planning  must  be  driven  by 
risk  management,  not  by  $  color  or  calendar! 


Figure  17-4.  Funding  Rule  for  DoD  System  Upgrades 


modification  process  because  it  has  serious  cost 
implications. 

Government  funding  rules:  As  Figure  17-4 
shows  the  use  of  government  funding  to  per¬ 
form  system  upgrades  has  restrictions.  The  pur¬ 
pose  of  the  upgrade  must  be  clear  and  justified 
in  the  planning  efforts. 

17.3  ROLES  AND  RESPONSIBILITIES 

Modification  management  is  normally  a  joint  gov¬ 
ernment  and  contractor  responsibility.  Though  any 
specific  system  upgrade  will  have  relationships 
established  by  the  conditions  surrounding  the  par¬ 
ticular  program,  government  responsibilities  would 
usually  include: 

•  Providing  a  clear  statement  of  system  require¬ 
ments, 

•  Planning  related  to  government  functions, 


•  Managing  the  functional  baseline  configura¬ 
tion,  and 

•  Verifying  that  requirements  are  satisfied. 

Contractor  responsibilities  are  established  by  the 
contract,  but  would  normally  include: 

•  Technical  planning  related  to  execution, 

•  Defining  the  new  performance  envelope, 

•  Designing  and  developing  modifications,  and 

•  Providing  evidence  that  changes  made  have 
modified  the  system  as  required. 

System  Engineering  Role 

The  systems  engineering  role  in  product  improve¬ 
ment  includes: 

•  Planning  for  system  change, 

•  Applying  the  systems  engineering  process, 


Managing  external  interfaces, 
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•  Managing  interface  changes, 

•  Identifying  and  using  interface  standards  which 
facilitate  continuing  change, 

•  Ensuring  life  cycle  management  is  imple¬ 
mented, 

•  Monitoring  the  need  for  system  modifications, 
and 

•  Ensuring  operations,  support  activities,  and 
early  field  results  are  considered  in  planning. 


17.4  SUMMARY  POINTS 

•  Complex  systems  do  not  usually  have  stagnant 
configurations. 

•  Planned  improvements  strategies  include 
evolutionary  acquisition,  pre-planned  product 
development,  and  open  systems. 

•  A  major  rebuild  should  be  treated  as  a  new 
system  development. 

•  Upgrading  an  existing  system  is  a  matter  of 
following  the  system  engineering  process,  with 
an  emphasis  on  configuration  and  interface 
management. 

•  Pay  attention  to  the  traps.  Upgrade  projects 
have  many. 
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OPEN  SYSTEM  APPROACH 


The  open  system  approach  is  a  business  and 
technical  approach  to  system  development  that 
results  in  systems  that  are  easier  to  change  or 
upgrade  by  component  replacement.  It  is  a  system 
development  logic  that  emphasizes  flexible 
interfaces  and  maximum  interoperability,  optimum 
use  of  commercial  competitive  products,  and 
enhanced  system  capacity  for  future  upgrade.  The 
value  of  this  approach  is  that  open  systems  have 
flexibility,  and  that  flexibility  translates  into 
benefits  that  can  be  recognized  from  business, 
management,  and  technical  perspectives. 

From  a  management  and  business  view,  the  open 
system  approach  directs  resources  to  a  more 
intensive  design  effort  with  the  expectation  of  a 
life  cycle  cost  reduction.  As  a  business  approach 
it  supports  the  DoD  policy  initiatives  of  CAIV, 
increased  competition,  and  use  of  commercial 
products.  It  is  a  technical  approach  that  emphasizes 
systems  engineering,  interface  control,  modular 


design,  and  design  for  upgrade.  As  a  technical 
approach  it  supports  the  engineering  goals  of 
design  flexibility,  risk  reduction,  configuration 
control,  long-term  supportability,  and  enhanced 
utility. 

Open  Systems  Initiative 

In  DoD  the  open  system  initiative  was  begun  as  a 
result  of  dramatic  changes  in  the  computer  industry 
that  afforded  significant  advantages  to  design  of 
C4ISR  and  IT  systems.  The  standardization 
achieved  by  the  computer  industry  allows  C4ISR 
and  IT  systems  to  be  designed  using  interface 
standards  to  select  off-the-shelf  components  to 
form  the  system.  This  is  achieved  by  using 
commercially-supported  specifications  and 
standards  for  specifying  system  interfaces  (ex¬ 
ternal  and  internal,  functional  and  physical),  prod¬ 
ucts,  practices,  and  tools.  An  open  system  is  one 
in  which  interfaces  are  fully  described  by  open 
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Figure  17-6.  Simplified  Computer  Resource  Reference  Model 


standards.1  An  open  system  approach  extends 
this  concept  further  by  using  modular  design 
and  interface  design  to  enhance  the  availability 
of  multiple  design  solutions,  especially  those 
reflecting  use  of  open  standards,  competitive 
commercial  components,  non-developmental 
items,  and  future  upgrade  capability. 

As  developed  in  the  C4ISR  and  IT  communities, 
the  open  system  approach  requires  the  design  of 
three  architectures:  operational,  technical,  and 
system. 

As  shown  in  Figure  17-5,  the  first  one  prepared  is 
an  operational  architecture  that  defines  the  tasks, 
operational  elements,  and  information  flows 
required  to  accomplish  or  support  an  operational 
function.  The  user  community  generates  the 
operational  concepts  that  form  an  operational 
architecture.  The  operational  architecture  is 
allusive.  It  is  not  a  specific  document  required 
to  be  developed  by  the  user  such  as  the  ORD; 
but  because  of  their  operational  nature,  the  user 
must  provide  the  components  of  the  operational 


architecture.  It  is  usually  left  to  the  developer  to 
assemble  and  structure  the  information  as  part 
of  the  system  definition  requirements  analysis. 
Once  the  operational  architecture  has  clearly 
defined  the  operational  need,  development  of 
a  system  architecture2  is  begun. 

The  (open)  system  architecture  is  a  set  of  descrip¬ 
tions,  including  graphics,  of  systems  and  intercon¬ 
nections  supporting  the  operational  functions 
described  in  the  operational  architecture.  Early  in 
the  (open)  system  architecture  development  a 
technical  architecture  is  prepared  to  establish  a  set 
of  rules,  derived  from  open  consensus-based 
industry  standards,  to  govern  the  arrangement, 
interaction,  and  interdependence  of  the  elements 
of  a  reference  model.  Reference  models  are  a 
common  conceptual  framework  for  the  type  of 
system  being  designed.  (A  simple  version  for 
computer  resources  is  shown  in  Figure  17-6.) 

The  technical  architecture  identifies  the  services, 
interfaces,  standards,  and  their  relationships;  and 
provides  the  technical  guidelines  upon  which 


1  Open  Standards  are  non-proprietary,  consensus-based  standards  widely  accepted  by  industry.  Examples  include  SAE,  IEEE,  and  ISO 
standards. 

2  This  system  architecture  typically  describes  the  end  product  but  not  the  enabling  products,  it  relies  heavily  on  interface  definitions  to 
describe  system  components. 
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engineering  specifications  are  based,  common 
building  blocks  are  built,  and  product  lines  are 
developed.  In  short,  the  technical  architecture 
becomes  a  design  requirement  for  developing  the 
system.  (The  purpose,  form,  and  function  of  the 
technical  architecture  is  similar  to  building  codes.) 

The  system  architecture  is  then  further  developed 
to  eventually  specify  component  performance  and 
interface  requirements.  These  are  then  used  to 
select  the  specific  commercial  components  that 
form  the  system  under  development.  This  process, 
called  an  implementation,  envisions  the  produc¬ 
tion  process  as  consisting  primarily  of  selecting 
components,  conformance  (to  the  interface  and 
performance  requirements)  management,  and 
assembly,  with  little  or  no  need  for  detailed  design 
fabrications. 

The  process  described  above  has  allowed  signifi¬ 
cant  achievements  in  computer-related  develop¬ 
ments.  Other  technical  fields  have  also  used  the 
open  system  design  approach  extensively.  (Com¬ 
mon  examples  are  the  electrical  outlets  in  your 
home  and  the  tire-to-wheel  interface  on  your  car). 
In  most  cases  the  process  is  not  as  well  defined  as 
it  is  in  the  current  digital  electronics  area.  A  con¬ 
sistent  successful  use  of  the  open  design  concept, 
in  and  outside  the  electronics  field,  requires  an 
understanding  of  how  this  process  relates  to  the 
activities  associated  with  systems  engineering 
management. 

Systems  Engineering  Management 

The  open  system  approach  impacts  all  three 
essential  elements  of  systems  engineering  man¬ 
agement:  systems  engineering  phasing,  the  sys¬ 
tems  engineering  process,  and  life  cycle  consider¬ 
ations.  It  requires  enhanced  interface  management 
in  the  systems  engineering  process,  and  requires 
specific  design  products  be  developed  prior  to  en¬ 
gineering-event  milestones.  The  open  systems 
approach  is  inherently  life  cycle  friendly.  It 
favorably  impacts  production  and  support  func¬ 
tions,  but  it  also  requires  additional  effort  to  assure 
life  cycle  conformance  to  interface  requirements. 


Open  Systems  Products  and 
SE  Development  Phasing 

A  system  is  developed  with  stepped  phases  that 
allow  an  understanding  of  the  operational  need  to 
eventually  evolve  into  a  design  solution.  Though 
some  tailoring  of  this  concept  is  appropriate,  the 
basic  phasing  (based  on  the  operational  concept 
preceding  the  system  description,  which  precedes 
the  preliminary  design,  which  precedes  the  detailed 
design)  is  necessary  to  coordinate  the  overall 
design  process  and  control  the  requirements  flow- 
down.  As  shown  by  Figure  17-7  the  open  system 
approach  blends  well  with  these  development 
phases. 

Concept  Studies  Phase:  Operational  Architecture 
The  initial  detailed  operational  concept,  includ¬ 
ing  operational  architectures,  should  be  a  user- 
community  output  (with  some  acquisition  engi¬ 
neering  assistance)  produced  during  the  concept 
exploration  phase  that  emphasizes  operational 
concepts  associated  with  various  material  solu¬ 
tions.  The  operational  concept  is  then  updated  as 
necessary  for  each  following  phase.  Analysis  of 
the  initial  operational  concept  should  be  a  key 
element  of  the  operational  view  output  of  the 
system  definition  phase  requirements  analysis.  An 
operational  architecture  developed  for  supporting 
the  system  description  should  be  complete,  com¬ 
prehensive,  and  clear;  and  verified  to  be  so  at  the 
Alternative  Systems  Review.  If  the  operational 
architecture  cannot  be  completed,  then  a  core 
operational  capability  must  be  developed  to 
establish  the  basis  for  further  development.  Where 
a  core  capability  is  used,  core  requirements  should 
be  complete  and  firm,  and  the  process  for  adding 
expanded  requirements  should  be  clear  and 
controlled. 

System  Definition  Phase 

System  interface  definitions,  such  as  the  technical 
architecture,  and  high-level  (open)  system  archi¬ 
tecture  should  be  complete  in  initial  form  at  the 
end  of  the  system  definition  phase  (along  with 
other  functional  baseline  documentation).  Success¬ 
ful  completion  of  these  items  is  required  to  per¬ 
form  the  preliminary  design,  and  they  should  be 
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Figure  17-7.  Phasing  of  Open  System  Development 


available  for  the  System  Functional  Review,  also 
referred  to  as  the  System  Definition  Review  or 
System  Design  Review.  The  open  system  docu¬ 
mentation  can  be  separate  or  incorporated  in  other 
functional  baseline  documentation.  The  criteria  for 
acceptance  should  be  established  in  the  systems 
engineering  management  plan  as  phase-exit  criteria. 

Preliminary  Design  Phase 

Along  with  other  allocated  baseline  documenta¬ 
tion,  the  interface  definitions  should  be  updated 
and  the  open-system  architecture  completed  by 
the  end  of  the  preliminary  design  effort.  This 
documentation  should  also  identify  the  proper  level 
of  openness  (that  is,  the  level  of  system  decompo¬ 
sition  at  which  the  open  interfaces  are  established) 
to  obtain  the  maximum  cost  and  logistic  advantage 
available  from  industry  practice. 

The  preliminary  design  establishes  performance- 
based  descriptions  of  the  system  components,  as 
well  as  the  interface  and  structure  designs  that 
integrate  those  components.  It  is  in  this  phase  that 
the  open  system  approach  has  the  most  impact. 


Interface  control  should  be  enhanced  and  fo¬ 
cused  on  developing  modular  designs  that  al¬ 
low  for  maximum  interchange  of  competitive 
commercial  products.  Review  of  the  technical 
architecture  (or  interface  definitions)  becomes 
a  key  element  of  requirements  analysis,  open 
system  focused  functional  partitioning  becomes 
a  key  element  of  functional  analysis  and  allo¬ 
cation,  iterative  analysis  of  modular  designs 
becomes  a  key  element  of  design  synthesis,  and 
conformance  management  becomes  a  key  ele¬ 
ment  of  verification.  Open  system  related  prod¬ 
ucts,  such  as  the  technical  architecture,  interface 
management  documentation,  and  conformance 
management  documentation,  should  be  key 
data  reviewed  at  the  Preliminary  Design  Review. 
Again,  the  criteria  for  acceptance  should  be  es¬ 
tablished  in  the  systems  engineering  manage¬ 
ment  plan  as  phase-exit  criteria. 

Detail  Design  Phase 

The  detail  design  phase  becomes  the  implementa¬ 
tion  for  those  parts  of  the  system  that  have  achieved 
open  system  status.  Conformance  management 
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becomes  a  significant  activity  as  commercial 
components  are  chosen  to  meet  performance 
and  interface  requirements.  Conformance  and 
interface  design  testing  becomes  a  driving  ac¬ 
tivity  during  verification  to  assure  an  open  sys¬ 
tem  or  subsystem  has  been  achieved  and  that 
components  selected  meet  interface  require¬ 
ments  and/or  standards. 

Systems  Engineering  Process 

The  systems  engineering  problem  solving  process 
consists  of  process  steps  and  loops  supported  by 
system  analysis  and  control  tools.  The  focus  of 
the  open  systems  engineering  process  is  compart¬ 
mentalized  design,  flexible  interfaces,  recognized 
interface  standards,  standard  components  with 
recognized  common  interfaces,  use  of  commer¬ 
cial  and  non-developmental  items,  and  an  increased 
emphasis  on  interface  control.  As  shown  by  Fig¬ 
ure  17-8,  the  open-system  approach  comple¬ 
ments  the  systems  engineering  process  to  pro¬ 
vide  an  upgradeable  design. 

Requirements  analysis  includes  the  review  and 
update  of  interface  standards  and  other  interface 


definitions  generated  as  output  from  previous 
systems  engineering  processes.  Functional 
analysis  and  allocation  focuses  on  functional 
partitioning  to  identify  functions  that  can  be 
performed  independent  of  each  other  in  order 
to  minimize  functional  interfaces.  Design  syn¬ 
thesis  focuses  on  modular  design  with  open 
interfaces,  use  of  open  standards  compliant 
commercial  products,  and  the  development  of 
performance  and  interface  specifications.  The 
verification  processes  include  conformance 
testing  to  validate  the  interface  requirements  are 
appropriate  and  to  verify  components  chosen 
to  implement  the  design  meet  the  interface  re¬ 
quirements.  Engineering  open  designs,  then, 
does  not  alter  the  fundamental  practices  within 
systems  engineering,  but,  rather,  provides  a  spe¬ 
cific  focus  to  the  activities  within  that  process. 

System  Engineering  Control: 

Interface  Management 

The  key  to  the  open  systems  engineering  pro¬ 
cess  is  interface  management.  Interface  manage¬ 
ment  should  be  done  in  a  more  formal  and  com¬ 
prehensive  manner  to  rigidly  identify  all  inter- 


Figure  17-8.  Open  System  Approach  to  the  Systems  Engineering  Process 


Chapter  17 


Product  Improvement  Strategies 


faces  and  control  the  flowdown  and  integration 
of  interface  requirements.  The  interfaces  become 
controlled  elements  of  the  baseline  equal  to  (or 
considered  part  of)  the  configuration.  Open  sys¬ 
tem  interface  management  emphasizes  the  cor¬ 
relation  of  interface  requirements  between  in¬ 
terfacing  systems.  (Do  those  designing  the  in¬ 
terfacing  systems  understand  the  interface  re¬ 
quirements  in  the  same  way?)  Computer-Aided 
System  Engineering  (CASE)  generated  schematic 
block  diagrams  can  be  used  to  track  interface 
design  activity. 

An  open  system  is  also  characterized  by  multiple 
design  solutions  within  the  interfaces  with  empha¬ 
sis  on  leveraging  best  commercial  practice.  The 
interface  management  effort  must  control  interface 
design  such  that  interfaces  specifically  chosen  for 
an  open  system  approach  are  designed  based  on 
the  following  priority: 

•  Open  standards  that  allow  competitive  products, 

•  Open  interface  design  that  allows  installation 
of  competitive  products  with  minimal  change, 

•  Open  interface  design  that  allows  minimal 
change  installation  of  commercial  or  NDI  prod¬ 
ucts  currently  or  planned  to  be  in  DoD  use,  and 
last, 

•  Unique  design  with  interfaces  designed  with 
upgrade  issues  considered. 

Note  that  these  are  clear  priorities,  not  options. 

Level  of  Openness 

The  level  at  which  the  interface  design  should  focus 
on  openness  is  also  a  consideration.  Each  system 
may  have  several  levels  of  openness  depending  on 
the  complexity  of  the  system  and  the  differences 
in  the  technology  within  the  system.  The  level 
chosen  to  define  the  open  interfaces  should  be 
supported  by  industry  and  be  consistent  with 
program  objectives.  For  example,  for  most  digi¬ 
tal  electronics  that  level  is  the  line-replaceable 
(LRU)  and  shop-replaceable  (SRU)  level.  On  the 
other  hand  the  Joint  Strike  Fighter  intends  to 


establish  openness  at  a  very  high  subsystem  level 
to  achieve  a  major  program  objective,  develop¬ 
ment  of  different  planes  using  common  building 
blocks  (which,  in  essence,  serve  as  the  refer¬ 
ence  model  for  the  family  of  aircraft.)  The  open 
system  approach  designed  segments  of  a  larger 
system  could  have  additional  openness  at  a  lower 
level.  For  example,  the  Advanced  Amphibious 
Assault  Vehicle  engine  compartment  is  an  open 
approach  design  allowing  for  different  engine 
installation  and  future  upgrade  capability.  On  a 
lower  level  within  the  compartment  the  fuel  fil¬ 
ters,  lines,  and  connectors  are  defined  by  open 
standard  based  interfaces.  Other  systems  will 
define  openness  at  other  levels.  Program  objec¬ 
tives  (such  as  interoperability,  upgrade  capabil¬ 
ity,  cost-effective  support,  affordability,  and  risk 
reduction)  and  industry  practice  (based  on  mar¬ 
ket  research)  drive  the  choice  of  the  level  of 
openness  that  will  best  assure  optimum  utility 
and  availability  of  the  open  system  approach. 

Life  Cycle  Considerations 

Life  cycle  integration  is  established  primarily 
through  the  use  of  integrated  teaming  that  com¬ 
bines  the  design  and  life  cycle  planning.  The  ma¬ 
jor  impacts  on  life  cycle  activity  include: 

•  Time  and  cost  to  upgrade  a  system  is  reduced. 
It  is  common  in  defense  systems,  which  have 
average  life  spans  in  excess  of  40  years,  that 
they  will  require  upgrade  in  their  life  due  to 
obsolescence  of  original  components,  threat 
increase,  and  technology  push  that  increases 
economy  or  performance.  (Most  commercial 
products  are  designed  for  a  significantly  shorter 
life  than  military  systems,  and  designs  that  rely 
on  these  commercial  products  must  expect  that 
original  commercial  components  will  not 
necessarily  be  available  throughout  the  system’s 
life  cycle.)  By  using  an  open  system  approach 
the  ability  to  upgrade  a  system  by  changing 
a  single  or  set  of  components  is  greatly  en¬ 
hanced.  In  addition,  the  open  system  approach 
eases  the  design  problem  of  replacing  the 
component,  thereby  reducing  the  cost  and 
schedule  of  upgrade,  which  in  turn  reduces 
the  operational  impact. 
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•  An  open  system  approach  enhances  the  use 
of  competitive  products  to  support  the  sys¬ 
tem.  This  flexibility  tends  to  reduce  the  cost 
associated  with  supply  support,  but  more  im¬ 
portantly  improves  component  and  parts 
availability. 

•  Conformance  management  becomes  a  part  of 
the  life  cycle  configuration  process.  Replace¬ 
ment  of  components  in  an  open  system  must 
be  more  controlled  because  the  government  has 
to  control  the  system  configuration  without  con¬ 
trolling  the  detail  component  configuration 
(which  will  come  from  multiple  sources,  all 
with  different  detail  configurations).  The  gov¬ 
ernment  must  expect  that  commercial  suppli¬ 
ers  will  control  the  design  of  their  components 
without  regard  to  the  government’s  systems. 
The  government  therefore  must  use  perfor¬ 
mance-  and  interface-based  specifications  to 
assure  the  component  will  provide  service 
equivalent  to  that  approved  through  the  ac¬ 
quisition  process.  Conformance  management 


is  the  process  that  tracks  the  interface  require¬ 
ments  through  the  life  cycle,  and  assures  that 
the  new  product  meets  those  requirements. 

Summary  Comments 

Open  system  design  is  not  only  compatible  with 
systems  engineering;  it  represents  an  approach  that 
enhances  the  overall  systems  engineering  effort. 
It  controls  interfaces  comprehensively,  provides 
interface  visibility,  reduces  risk  through  multiple 
design  solutions,  and  insists  on  life  cycle  inter¬ 
face  control.  This  emphasis  on  interface  identifi¬ 
cation  and  control  improves  systems  engineers’ 
capability  to  integrate  the  system,  probably  one  of 
the  hardest  jobs  they  have.  It  also  improves  the 
tracking  of  interface  requirements  flow  down, 
another  key  job  of  the  systems  engineer.  Perhaps 
most  importantly,  this  rigorous  interface  manage¬ 
ment  improves  systems  engineers’  ability  to 
correctly  determine  where  commercial  items  can 
be  properly  used. 
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18.1  INTEGRATED  DEVELOPMENT 

DoD  has,  for  years,  required  that  system  designs 
be  integrated  to  balance  the  conflicting  pressure 
of  competing  requirements  such  as  performance, 
cost,  supportability,  producibility,  and  testability. 
The  use  of  multi-disciplinary  teams  is  the  approach 
that  both  DoD  and  industry  increasing  have  taken 
to  achieve  integrated  designs.  Teams  have  been 
found  to  facilitate  meeting  cost,  performance,  and 
other  objectives  from  product  concept  through 
disposal. 

The  use  of  multi-disciplinary  teams  in  design  is 
known  as  Integrated  Product  and  Process  Devel¬ 
opment,  simultaneous  engineering,  concurrent 
engineering,  Integrated  Product  Development, 
Design-Build,  and  other  proprietary  and  non-pro¬ 
prietary  names  expressing  the  same  concept.  (The 
DoD  use  of  the  term  Integrated  Product  and  Pro¬ 
cess  Development  (IPPD)  is  a  wider  concept  that 
includes  the  systems  engineering  effort  as  an  ele¬ 
ment.  The  DoD  policy  is  explained  later  in  this 
chapter.)  Whatever  name  is  used,  the  fundamental 
idea  involves  multi-functional,  integrated  teams 
(preferably  co-located),  that  jointly  derive  require¬ 
ments  and  schedules  that  place  equal  emphasis  on 
product  and  process  development.  The  integration 
requires: 

•  Inclusion  of  the  Eight  Primary  Functions  in  the 
team(s)  involved  in  the  design  process, 

•  Technical  process  specialties  such  as  quality, 
risk  management,  safety,  etc.,  and 


Benefits 

The  expected  benefits  from  team-based  integration 
include: 

•  Reduced  rework  in  design,  manufacturing, 
planning,  tooling,  etc., 

•  Improved  first  time  quality  and  reduction  of 
product  variability, 

•  Reduced  cost  and  cycle  time, 

•  Reduced  risk, 

•  Improved  operation  and  support,  and 

•  General  improvement  in  customer  satisfaction 
and  product  quality  throughout  its  life  cycle. 

Characteristics 

The  key  attributes  that  characterize  a  well 
integrated  effort  include: 

•  Customer  focus, 

•  Concurrent  development  of  products  and 
processes, 

•  Early  and  continuous  life  cycle  planning, 

•  Maximum  flexibility  for  optimization, 

•  Robust  design  and  improved  process  capability, 

•  Event-driven  scheduling, 

•  Multi-disciplinary  teamwork, 


Business  processes  (usually  in  an  advisory 
capacity)  such  as,  finance,  legal,  contracts,  and 
other  non-technical  support. 
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•  Empowerment, 

•  Seamless  management  tools,  and 

•  Proactive  identification  and  management  of 
risk. 

Organizing  for  System  Development 

Most  DoD  program  offices  are  part  of  a  Program 
Executive  Office  (PEO)  organization  that  is  usu¬ 
ally  supported  by  a  functional  organization,  such 
as  a  systems  command.  Contractors  and  other  gov¬ 
ernment  activities  provide  additional  necessary 
support.  Establishing  a  system  development  orga¬ 
nization  requires  a  network  of  teams  that  draw  from 
all  these  organizations.  This  network,  sometimes 
referred  to  as  the  enterprise,  represents  the  inter¬ 
ests  of  all  the  stakeholders  and  provides  vertical 
and  horizontal  communications. 

These  integrated  teams  are  structured  using  the 
WBS  and  designed  to  provide  the  maximum  ver¬ 


tical  and  horizontal  communication  during  the 
development  process.  Figure  18-1  shows  how  team 
structuring  is  usually  done.  At  the  system  level 
there  is  usually  a  management  team  and  a  design 
team.  The  management  team  would  normally  con¬ 
sist  of  the  government  and  contractor  program 
managers,  the  deputy  program  manager(s),  possi¬ 
bly  the  contractor  CEO,  the  contracting  officer, 
major  advisors  picked  by  the  program  manager, 
the  system  design  team  leader,  and  other  key  mem¬ 
bers  of  the  system  design  team.  The  design  team 
usually  consists  of  the  first-level  subsystem  and 
life-cycle  integrated  team  leaders. 

The  next  level  of  teams  is  illustrated  on  Figure 
18-1  as  either  product  or  process  teams.  These 
teams  are  responsible  for  designing  system  seg¬ 
ments  (product  teams)  or  designing  the  support¬ 
ing  or  enabling  products  (process  teams).  At  this 
level  the  process  teams  are  coordinating  the  sys¬ 
tem  level  process  development.  For  example, 
the  support  team  will  integrate  the  supportabil- 
ity  analysis  from  the  parts  being  generated  in 
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lower-level  design  and  support  process  teams. 
Teams  below  this  level  continue  the  process  at  a 
lower  level  of  decomposition.  Teams  are  formed 
only  to  the  lowest  level  necessary  to  control  the 
integration.  DoD  team  structures  rarely  extend 
lower  than  levels  three  or  four  on  the  WBS,  while 
contractor  teams  may  extend  to  lower  levels,  de¬ 
pending  on  the  complexities  of  the  project  and 
the  approach  favored  by  management. 

The  team  structure  shown  by  Figure  18-1  is  a 
hierarchy  that  allows  continuous  vertical  commu¬ 
nication.  This  is  achieved  primarily  by  having  the 
team  leaders,  and,  if  appropriate,  other  key 
members  of  a  team,  be  team  members  of  the  next 
highest  team.  In  this  manner  the  decisions  of  the 
higher  team  is  immediately  distributed  and 
explained  to  the  next  team  level,  and  the  decisions 
of  the  lower  teams  are  presented  to  the  higher  team 
on  a  regular  basis.  Through  this  method  decisions 
of  lower-level  teams  follow  the  decision  mak¬ 
ing  of  higher  teams,  and  the  higher-level  teams’ 


decisions  incorporate  the  concerns  of  lower-level 
teams. 

The  normal  method  to  obtain  horizontal  commu¬ 
nication  is  shown  in  Figure  18-2.  At  least  one  team 
member  from  the  Product  A  Team  is  also  a  member 
of  the  Integration  and  Test  Team.  This  member 
would  have  a  good  general  knowledge  of  both 
testing  and  Product  A.  The  member’s  job  would 
be  to  assist  the  two  teams  in  designing  their  end 
or  enabling  products,  and  in  making  each  under¬ 
stand  how  their  decisions  would  impact  the  other 
team.  Similarly,  the  member  that  sits  on  both 
Product  A  and  B  teams  would  have  to  understand 
the  both  technology  and  the  interface  issues 
associated  with  both  items. 

The  above  is  an  idealized  case.  Each  type  of  sys¬ 
tem,  each  type  of  contractor  organization,  and  each 
level  of  available  resources  requires  a  tailoring  of 
this  structure.  With  each  phase  the  focus  and  the 
tasks  change  and  so  should  the  structure.  As  phases 


Figure  18-2.  Cross  Membership 
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are  transited,  the  enterprise  structure  and  team 
membership  should  be  re-evaluated  and  updated. 

18.2  INTEGRATED  TEAMS 

Integrated  teams  are  composed  of  representatives 
from  all  appropriate  primary  functional  disciplines 
working  together  with  a  team  leader  to: 

•  Design  successful  and  balanced  products, 

•  Develop  the  configuration  for  successful  life- 
cycle  control, 

•  Identify  and  resolve  issues,  and 

•  Make  sound  and  timely  decisions. 

The  teams  follow  the  disciplined  approach  of  the 
systems  engineering  process  starting  with  require¬ 
ments  analysis  through  to  the  development  of  con¬ 
figuration  baselines  as  explained  earlier  in  this 
book.  The  system-level  design  team  should  be 
responsible  for  systems  engineering  management 
planning  and  execution.  The  system-level  manage¬ 
ment  team,  the  highest  level  program  IPT,  is 
responsible  for  acquisition  planning,  resource 
allocation,  and  management.  Lower-level  teams 
are  responsible  for  planning  and  executing  their 
own  processes. 

Team  Organization 

Good  teams  do  not  just  happen;  they  are  the  result 
of  calculated  management  decisions  and  actions. 
Concurrent  with  development  of  the  enterprise 
organization  discussed  above,  each  team  must  also 
be  developed.  Basically  the  following  are  key 
considerations  in  planning  for  a  team  within  an 
enterprise  network: 

•  The  team  must  have  appropriate  representation 
from  the  primary  functions,  technical  special¬ 
ties,  and  business  support, 

•  There  must  be  links  to  establish  vertical  and 
horizontal  communication  in  the  enterprise, 


•  You  should  limit  over-uses  of  cross  member¬ 
ship.  Limit  membership  on  three  or  four  teams 
as  a  rough  rule  of  thumb  for  the  working  level, 
and 

•  Ensure  appropriate  representation  of  govern¬ 
ment,  contractor,  and  vendors  to  assure 
integration  across  key  organizations. 

Team  Development 

When  teams  are  formed  they  go  through  a  series 
of  phases  before  a  synergistic  self-actuating  team 
is  evolved.  These  phases  are  commonly  referred 
to  as  forming,  storming,  norming  and  perform¬ 
ing.  The  timing  and  intensity  of  each  phase  will 
depend  on  the  team  size,  membership  personality, 
effectiveness  of  the  team  building  methods 
employed,  and  team  leadership.  The  team  leaders 
and  an  enterprise-level  facilitator  provide 
leadership  during  the  team  development. 

Forming  is  the  phase  where  the  members  are  in¬ 
troduced  to  their  responsibilities  and  other  mem¬ 
bers.  During  this  period  members  will  tend  to  need 
a  structured  situation  with  clarity  of  purpose  and 
process.  If  members  are  directed  during  this  ini¬ 
tial  phase,  their  uncertainty  and  therefore  appre¬ 
hension  is  reduced.  Facilitators  controlling  the 
team  building  should  give  the  members  rules  and 
tasks,  but  gradually  reduce  the  level  of  direction 
as  the  team  members  begin  to  relate  to  each  other. 
As  members  become  more  familiar  with  other 
members,  the  rules,  and  tasks,  they  become  more 
comfortable  in  their  environment  and  begin  to 
interact  at  a  higher  level. 

This  starts  the  storming  phase.  Storming  is  the 
conflict  brought  about  by  interaction  relating  to 
the  individuals'  manner  of  dealing  with  the  team 
tasks  and  personalities.  Its  outcome  is  members 
who  understand  the  way  they  have  to  act  with  other 
members  to  accomplish  team  objectives.  The 
dynamics  of  storming  can  be  very  complex  and 
intense,  making  it  the  critical  phase.  Some  teams 
will  go  through  it  quickly  without  a  visible  ripple, 
others  will  be  loud  and  hot,  and  some  will  never 
emerge  from  this  phase.  The  team  building  facili¬ 
tators  must  be  alert  to  dysfunctional  activity. 
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Members  may  need  to  be  removed  or  teams 
reorganized.  Facilitators  during  this  period  must 
act  as  coaches,  directing  but  in  a  personal  collabo¬ 
rative  way.  They  should  also  be  alert  for  members 
that  are  avoiding  storming,  because  the  team  will 
not  mature  if  there  are  members  who  are  not 
personally  committed  to  participate  in  it. 

Once  the  team  has  learned  to  interact  effectively  it 
begins  to  shape  its  own  processes  and  become  more 
effective  in  joint  tasks.  It  is  not  unusual  to  see  some 
reoccurrence  of  storming,  but  if  the  storming  phase 
was  properly  transitioned  these  incidences  should 
be  minor  and  easily  passed.  In  this  phase,  norming, 
the  team  building  facilitators  become  a  facilitator 
to  the  team — not  directing,  but  asking  penetrat¬ 
ing  questions  to  focus  the  members.  They  also 
monitor  the  teams  and  correct  emerging  prob¬ 
lems. 

As  the  team  continues  to  work  together  on  their 
focused  tasks,  their  performance  improves  until 
they  reach  a  level  of  self-actuation  and  quality 
decision  making.  This  phase,  performing,  can  take 
a  while  to  reach,  18  months  to  two  years  for  a 
system-level  design  team  would  not  be  uncommon. 
During  the  performing  stage,  the  team  building 
facilitator  monitors  the  teams  and  corrects 
emerging  problems. 

At  the  start  of  a  project  or  program  effort,  team 
building  is  commonly  done  on  an  enterprise  basis 
with  all  teams  brought  together  in  a  team-build¬ 
ing  exercise.  There  are  two  general  approaches  to 
the  exercise: 

•  A  team-learning  process  where  individuals  are 
given  short  but  focused  tasks  that  emphasize 
group  decision,  trust,  and  the  advantages  of 
diversity. 

•  A  group  work-related  task  that  is  important  but 
achievable,  such  as  a  group  determination  of 
the  enterprise  processes,  including  identifying 
and  removing  non-value  added  traditional 
processes. 

Usually  these  exercises  allow  the  enterprise  to  pass 
through  most  of  the  storming  phase  if  done  cor¬ 


rectly.  Three  weeks  to  a  month  is  reasonable  for 
this  process,  if  the  members  are  in  the  same  lo¬ 
cation.  Proximity  does  matter  and  the  team  build¬ 
ing  and  later  team  performance  are  typically  bet¬ 
ter  if  the  teams  are  co-located. 


18.3  TEAM  MAINTENANCE 

Teams  can  be  extremely  effective,  but  they  can  be 
fragile.  The  maintenance  of  the  team  structure  is 
related  to  empowerment,  team  membership  issues, 
and  leadership. 

Empowerment 

The  term  empowerment  relates  to  how  responsi¬ 
bilities  and  authority  is  distributed  throughout  the 
enterprise.  Maintenance  of  empowerment  is 
important  to  promote  member  ownership  of  the 
development  process.  If  members  do  not  have 
personal  ownership  of  the  process,  the  effective¬ 
ness  of  the  team  approach  is  reduced  or  even 
neutralized.  The  quickest  way  to  destroy  partici¬ 
pant  ownership  is  to  direct,  or  even  worse,  over¬ 
turn  solutions  that  are  properly  the  responsibil¬ 
ity  of  the  team.  The  team  begins  to  see  that  the 
responsibility  for  decisions  is  at  a  higher  level 
rather  than  at  their  level,  and  their  responsibility 
is  to  follow  orders,  not  solve  problems. 

Empowerment  requires: 

•  The  flow  of  authority  through  the  hierarchy  of 
teams,  not  through  personal  direction  (irrespec¬ 
tive  of  organizational  position.)  Teams  should 
have  clear  tasking  and  boundaries  established 
by  the  higher-level  teams. 

•  Responsibility  for  decision  making  to  be 
appropriate  for  the  level  of  team  activity.  This 
requires  management  and  higher-level  teams  to 
be  specific,  clear,  complete,  and  comprehen¬ 
sive  in  establishing  focus  and  tasking,  and 
in  specifying  what  decisions  must  be  coordi¬ 
nated  with  higher  levels.  They  should  then  avoid 
imposing  or  overturning  decisions  more 
properly  in  the  realm  of  a  lower  level. 
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•  Teams  at  each  level  be  given  a  clear  under¬ 
standing  of  their  duties  and  constraints.  Within 
the  bounds  of  those  constraints  and  assigned 
duties  members  should  have  autonomy. 
Higher-  level  teams  and  management  either 
accept  their  decisions,  or  renegotiate  the  un¬ 
derstanding  of  the  task. 

Membership  Issues 

Another  maintenance  item  of  import  is  team  mem¬ 
ber  turnover.  Rotation  of  members  is  a  fact  of  life, 
and  a  necessary  process  to  avoid  teams  becoming 
too  closed.  However,  if  the  team  has  too  fast  a 
turnover,  or  new  members  are  not  fully  assimi¬ 
lated,  the  team  performance  level  will  decline  and 
possibly  revert  to  storming.  The  induction  process 
should  be  a  team  responsibility  that  includes  the 
immediate  use  of  the  new  team  member  in  a  jointly 
performed,  short  term,  easily  achievable,  but  im¬ 
portant  task. 

Teams  are  responsible  for  their  own  performance, 
and  therefore  should  have  significant,  say  over 
the  choice  of  new  members.  In  addition  teams 
should  have  the  power  to  remove  a  member; 
however,  this  should  be  preceded  by  identifica¬ 
tion  of  the  problem  and  active  intervention  by 
the  facilitator.  Removal  should  be  a  last  resort. 

Awards  for  performance  should,  where  possible, 
be  given  to  the  team  rather  than  individuals  (or 
equally  to  all  individuals  on  the  team).  This 
achieves  several  things:  it  establishes  a  team  focus, 
shows  recognition  of  the  team  as  a  cohesive  force, 
recognizes  that  the  quality  of  individual  effort  is 
at  least  in  part  due  to  team  influence,  reinforces 
the  membership’s  dedication  to  team  objectives, 
and  avoids  team  member  segregation  due  to  uneven 
awards.  Some  variation  on  this  theme  is  appropri¬ 
ate  where  different  members  belong  to  different 
organizations,  and  a  common  award  system  does 
not  exist.  The  system-level  management  team 
should  address  this  issue,  and  where  possible 
assure  equitable  awards  are  given  team  members. 
A  very  real  constraint  on  cash  awards  in  DoD 
rises  in  the  case  of  teams  that  include  both  civil¬ 
ian  and  military  members.  Military  members  can¬ 
not  be  given  cash  awards,  while  civilians  can. 


Consequently,  managers  must  actively  seek  ways 
to  reward  all  team  members  appropriately, 
leaving  no  group  out  at  the  expense  of  others. 

Leadership 

Leadership  is  provided  primarily  by  the  organiza¬ 
tional  authority  responsible  for  the  program,  the 
enterprise  facilitator,  and  the  team  leaders.  In  a 
DoD  program,  the  organizational  leaders  are 
usually  the  Program  Manager  and  contractor  senior 
manager.  These  leaders  set  the  tone  of  the  enter¬ 
prise  adherence  to  empowerment,  the  focus  of  the 
technical  effort,  and  the  team  leadership  of  the 
system  management  team.  These  leaders  are 
responsible  to  see  that  the  team  environment  is 
maintained.  They  should  coordinate  their  action 
closely  with  the  facilitator. 

Facilitators 

Enterprises  that  have  at  least  one  facilitator  find 
that  team  and  enterprise  performance  is  easier  to 
maintain.  The  facilitator  guides  the  enterprise 
through  the  team  building  process,  monitors  the 
team  network  through  metrics  and  other  feedback, 
and  makes  necessary  corrections  through 
facilitation.  The  facilitator  position  can  be: 

•  A  separate  position  in  the  contractor 
organization, 

•  Part  of  the  responsibilities  of  the  government 
systems  engineer  or  contractor  project  manager, 
or 

•  Any  responsible  position  in  the  first  level  below 
the  above  that  is  related  to  risk  management. 

Obviously  the  most  effective  position  would  be 
one  that  allows  the  facilitator  to  concentrate  on 
the  teams’  performance.  Enterprise  level  facilita¬ 
tors  should  have  advanced  facilitator  training  and 
(recommended)  at  least  a  year  of  mentored  expe¬ 
rience.  Facilitators  should  also  have  significant 
broad  experience  in  the  technical  area  related  to 
the  development. 
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Team  Leaders 

The  team  leaders  are  essential  for  providing  and 
guiding  the  team  focus,  providing  vertical  com¬ 
munication  to  the  next  level,  and  monitoring  the 
team’s  performance.  Team  leaders  must  have  a 
clear  picture  of  what  constitutes  good  performance 
for  their  team.  They  are  not  supervisors,  though  in 
some  organizations  they  may  have  supervisory 
administrative  duties.  The  leader’s  primary  pur¬ 
pose  is  to  assure  that  the  environment  is  present 
that  allows  the  team  to  perform  at  its  optimum 
level — not  to  direct  or  supervise. 

The  team  leader’s  role  includes  several  difficult 
responsibilities: 

•  Taking  on  the  role  of  coach  as  the  team  forms, 

•  Facilitating  as  the  team  becomes  self- 
sustaining, 

•  Sometimes  serving  as  director.  (Only  when  a 
team  has  failed,  needs  refocus  or  correction, 
and  is  done  with  the  facilitator), 

•  Providing  education  and  training  for  members, 

•  Facilitating  team  learning, 

•  Representing  the  team  to  upper  management 
and  the  next  higher-level  team,  and 

•  Facilitating  team  disputes. 

Team  leaders  should  be  trained  in  basic  facilitator 
principles.  This  training  can  be  done  in  about  a 
week,  and  there  are  numerous  training  facilities 
or  companies  that  can  offer  it. 

4.4  TEAM  PROCESSES 

Teams  develop  their  processes  from  the  principles 
of  system  engineering  management  as  presented 
earlier  in  the  book.  The  output  of  the  teams  is  the 
design  documentation  associated  with  products 
identified  on  the  system  architecture,  including  both 
end  product  components  and  enabling  products. 


Teams  use  several  tools  to  enhance  their  pro¬ 
ductivity  and  improve  communication  among  en¬ 
terprise  members.  Some  examples  are: 

•  Constructive  modeling  (CAD/CAE/  CAM/ 
CASE)  to  enhance  design  understanding  and 
control, 

•  Trade-off  studies  and  prioritization, 

•  Event-driven  schedules, 

•  Prototyping, 

•  Metrics,  and  most  of  all 

•  Integrated  membership  that  represents  the  life 
cycle  stakeholders. 

Integrated  Team  Rules 

The  following  is  a  set  of  general  rules  that  should 
guide  the  activities  and  priorities  of  teams  in  a 
system  design  environment: 

•  Design  results  must  be  communicated  clearly, 
effectively,  and  timely. 

•  Design  results  must  be  compatible  with  initially 
defined  requirements. 

•  Continuous  “up-the-line”  communication  must 
be  institutionalized. 

•  Each  member  needs  to  be  familiar  with  all 
system  requirements. 

•  Everyone  involved  in  the  team  must  work  from 
the  same  database. 

•  Only  one  member  of  the  team  has  the  author¬ 
ity  to  make  changes  to  one  set  of  master 
documentation. 

•  All  members  have  the  same  level  of  authority 
(one  person,  one  vote). 

•  Team  participation  is  consistent,  success- 
oriented,  and  proactive. 
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•  Team  discussions  are  open  with  no  secrets. 

•  Team  member  disagreements  must  be  reasoned 
disagreement  (alternative  plan  of  action  versus 
unyielding  opposition). 

•  Trade  studies  and  other  analysis  techniques  are 
used  to  resolve  issues. 

•  Issues  are  raised  and  resolved  early. 

•  Complaints  about  the  team  are  not  voiced  out¬ 
side  the  team.  Conflicts  must  be  resolved 
internally. 

Guidelines  for  Meeting  Management 

Even  if  a  team  is  co-located  as  a  work  unit,  regular 
meetings  will  be  necessary.  These  meetings  and 
their  proper  running  become  even  more  important 
if  the  team  is  not  co-located  and  the  meeting  is  the 
primary  means  of  one-on-one  contact.  A  well  run 
technical  meeting  should  incorporate  the  following 
considerations: 

•  Meetings  should  be  held  only  for  a  specific 
purpose  and  a  projected  duration  should  be 
targeted. 

•  Advance  notice  of  meetings  should  normally 
be  at  least  two  weeks  to  allow  preparation  and 
communication  between  members. 

•  Agendas,  including  time  allocations  for  topics 
and  supportive  material  should  be  distributed 
no  less  than  three  business  days  before  the  team 
meeting.  The  objective  of  the  meeting  should 
be  clearly  defined. 

•  Stick  to  the  agenda  during  the  meeting.  Then 
cover  new  business.  Then  review  action  items. 

•  Meeting  summaries  should  record  attendance, 
document  any  decision  or  agreements  reached, 
document  action  items  and  associated  due- 
dates,  provide  a  draft  agenda  for  the  next 
meeting,  and  frame  issues  for  higher-level 
resolution. 


•  Draft  meeting  summaries  should  be  provided 
to  members  within  one  working  day  of  the 
meeting.  A  final  summary  should  be  issued 
within  two  working  days  after  the  draft 
comments  deadline. 


18.5  BARRIERS  TO  INTEGRATION 

There  are  numerous  barriers  to  building  and  main¬ 
taining  a  well  functioning  team  organization,  and 
they  are  difficult  to  overcome.  Any  one  of  these 
barriers  can  negate  the  effectiveness  of  an  inte¬ 
grated  development  approach.  Common  barriers 
include: 

•  Lack  of  top  management  support, 

•  Team  members  not  empowered, 

•  Lack  of  access  to  a  common  database, 

•  Lack  of  commitment  to  a  cultural  change, 

•  Functional  organization  not  fully  integrated  into 
a  team  process, 

•  Lack  of  planning  for  team  effort, 

•  Staffing  requirements  conflict  with  teams, 

•  Team  members  not  collocated, 

•  Insufficient  team  education  and  training, 

•  Lessons  learned  and  successful  practices  not 
shared  across  teams, 

•  Inequality  of  team  members, 

•  Lack  of  commitment  based  on  perceived  un¬ 
certainty, 

•  Inadequate  resources,  and 

•  Lack  of  required  expertise  on  either  the  part  of 
the  contractor  or  government. 
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Breaking  Barriers 

Common  methods  to  combat  barriers  include: 

•  Education  and  training,  and  then  more  educa¬ 
tion  and  training:  it  breaks  down  the  uncertainty 
of  change,  and  provides  a  vision  and  method 
for  success. 

•  Use  a  facilitator  not  only  to  build  and  maintain 
teams,  but  also  to  observe  and  advise  manage¬ 
ment. 

•  Obtain  management  support  up  front.  Manage¬ 
ment  must  show  leadership  by  managing  the 
teams’  environment  rather  than  trying  to 
manage  people. 

•  Use  a  common  database  open  to  all  enterprise 
members. 

•  Establish  a  network  of  teams  that  integrates  the 
design  and  provides  horizontal  and  vertical 
communication. 

•  Establish  a  network  that  does  not  over-tax  avail¬ 
able  resources.  Where  a  competence  is  not  avail¬ 
able  in  the  associated  organizations,  hire  it 
through  a  support  contractor. 


•  Where  co-location  is  not  possible  have  regular 
working  sessions  of  several  days  duration.  Tele¬ 
communications,  video  conferencing,  and  other 
technology  based  techniques  can  also  go  far  to 
alleviate  the  problems  of  non-collocation. 

Summary  Comments 

•  Integrating  system  development  is  a  systems 
engineering  approach  that  integrates  all 
essential  primary  function  activities  through  the 
use  of  multi-disciplinary  teams,  to  optimize  the 
design,  manufacturing  and  supportability 
processes. 

•  Team  building  goes  through  four  phases: 
forming,  storming,  norming,  and  performing. 

•  Key  leadership  positions  in  a  program  network 
of  teams  are  the  program  manager,  facilitator, 
and  team  leaders. 

•  A  team  organization  is  difficult  to  build  and 
maintain.  It  requires  management  attention  and 
commitment  over  the  duration  of  the  teams 
involved. 


165 


Systems  Engineering  Fundamentals 


Chapter  18 


SUPPLEMENT  A 

IPPD  - 

A  DOD  MANAGEMENT  PROCESS 


The  DoD  policy  of  Integrated  Product  and 
Process  Development  (IPPD)  is  a  broad  view  of 
integrated  system  development  which  includes 
not  only  systems  engineering,  but  other  areas 
involved  in  formal  decision  making  related  to 
system  development.  DoD  policy  emphasizes 
integrated  management  at  and  above  the  Pro¬ 
gram  Manager  (PM)  level.  It  requires  IPPD  at 
the  systems  engineering  level,  but  does  not  di¬ 
rect  specific  organizational  structures  or  proce¬ 
dures  in  recognition  of  the  need  to  design  a  tai¬ 
lored  IPPD  process  to  every  individual  situa¬ 
tion. 

Integrated  Product  Teams 

One  of  the  key  IPPD  tenets  is  multi-disciplinary 
integration  and  teamwork  achieved  through  the  use 
of  Integrated  Product  Teams  (IPTs).  While  IPTs 
may  not  be  the  best  solution  for  every  manage¬ 
ment  situation,  the  requirement  to  produce  inte¬ 
grated  designs  that  give  consideration  to  a  wide 
array  of  technical  and  business  concerns  leads  most 
organizations  to  conclude  that  IPTs  are  the  best 
organizational  approach  to  systems  management. 
PMs  should  remember  that  the  participation  of  a 
contractor  or  a  prospective  contractor  on  a  IPT 
should  be  in  accordance  with  statutory  require¬ 
ments,  such  as  procurement  integrity  rules.  The 
service  component’s  legal  advisor  must  review 
prospective  contractor  involvement  on  IPTs.  To 
illustrate  issues  the  government-contractor  team 
arrangement  raises,  the  text  box  at  the  end  of  this 
section  lists  nine  rules  developed  for  government 
members  of  the  Advanced  Amphibious  Assault 
Vehicle  (AAAV)  design  IPTs. 


The  Secretary  of  Defense  has  directed  that  DoD 
perform  oversight  and  review  by  using  IPTs.  These 
IPTs  function  in  a  spirit  of  teamwork  with 
participants  empowered  and  authorized,  to  the 
maximum  extent  possible,  to  make  commit¬ 
ments  for  the  organization  or  the  functional  area 
they  represent.  IPTs  are  composed  of  represen¬ 
tatives  from  all  appropriate  functional  disciplines 
working  together  to  build  successful  programs 
and  enabling  decision  makers  to  make  the  right 
decisions  at  the  right  time. 

DoD  IPT  Structure 

The  DoD  oversight  function  is  accomplished 
through  a  hierarchy  of  teams  that  include  levels  of 
management  from  DoD  to  the  program  level.  There 
are  three  basic  levels  of  IPTs:  the  Overaching  IPT 
(OIPT),  the  Working  IPTs  (WIPT),  and  Program 
IPTs  with  the  focus  and  responsibilities  as  shown 
by  Figure  18-3.  For  each  ACAT  I  program,  there 
will  be  an  OIPT  and  at  least  one  WIPT.  WIPTs 
will  be  developed  for  particular  functional  topics, 
e.g.  test,  cost/performance,  contracting,  etc.  An 
Integrating  IPT  (IIPT)  will  coordinate  WIPT  efforts 
and  cover  all  topics  not  otherwise  assigned  to 
another  IPT.  These  teams  are  structurally  organized 
as  shown  on  Figure  18-4. 

Overarching  IPT  (OIPT) 

The  OIPT  is  a  DoD  level  team  whose  primary  re¬ 
sponsibility  is  to  advise  the  Defense  Acquisition 
Executive  on  issues  related  to  programs  managed 
at  that  level.  The  OIPT  membership  is  made  up  of 
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Organization 

Teams 

Focus 

Participant 

Responsibilities 

OSD  and 
Components 

01 PT* 

•  Strategic  Guidance 

•  Tailoring 

•  Program  Assessment 

•  Resolve  Issues  Elevated  by  WIPTs 

•  Program  Success 

•  Functional  Area  Leadership 

•  Independent  Assessment 

•  Issue  Resolution 

WIPTs* 

•  Planning  for  Program  Success 

•  Opportunities  for  Acquisition 

Reform  (e.g.  innovation,  streamlining 

•  Identify/Resolve  Program  Issues 

•  Program  Status 

•  Functional  Knowledge  and  Experience 

•  Empowered  Contribution 

•  Recom.’s  for  Program  Success 

•  Communicate  Status  and  Unresolved 
Issues 

Program 

Teams  and 

System 

Contractors 

Program 

IPTs** 

•  Program  Execution 

•  Identify  and  Implement  Acquisition 
Reform 

•  Manage  Complete  Scope  of  Program 
Resources,  and  Risk 

•  Integrate  Government  and  Contractor 
Efforts  for  Report  Program  Status  and 
Issues 

*  Covered  in  “Rules  of  the  Road” 

**  Covered  in  “Guide  to  Implementation  and  Management  of  IPPD  in  DoD  Acquisition 

Figure  18-3.  Focus  and  Responsibilities  of  IPTs 
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the  principals  that  are  charged  with  responsibil¬ 
ity  for  the  many  functional  offices  at  the  Office 
of  the  Secretary  of  Defense  (OSD). 

The  OIPT  provides: 

•  Top-level  strategic  guidance, 

•  Functional  area  leadership, 

•  Forum  for  issue  resolution, 

•  Independent  assessment  to  the  MDA, 

•  Determine  decision  information  for  next 
milestone  review,  and 

•  Provide  approval  of  the  WIPT  structures  and 
resources. 

Working-Level  IPT  (WIPT) 

The  WIPTs  may  be  thought  of  as  teams  that  link 
the  program  manager  to  the  OIPT.  WIPTs  are  typi¬ 
cally  functionally  specialized  teams  (test,  cost- 
performance,  etc.).  The  PM  is  the  designated  head 
of  the  WIPT,  and  membership  typically  includes 
representation  from  various  levels  from  the  pro¬ 
gram  to  OSD  staff.  The  principal  functions  of  the 
WIPT  are  to  advise  the  PM  is  the  area  of  special¬ 
ization  and  to  advise  the  OIPT  of  program  status. 


The  duties  of  the  WIPT  include: 

•  Assisting  the  PM  in  developing  strategies  and 
in  program  planning,  as  requested  by  the  PM, 

•  Establishing  IPT  plan  of  action  and  mile¬ 
stones, 

•  Proposing  tailored  document  and  milestone 
requirements, 

•  Reviewing  and  providing  early  input  to 
documents, 

•  Coordinating  WIPT  activities  with  the  OIPT 
members, 

•  Resolving  or  evaluating  issues  in  a  timely 
manner,  and 

•  Obtaining  principals’  concurrence  with 
applicable  documents  or  portions  of  documents. 

Program  IPTs 

Program  IPTs  are  teams  that  perform  the  program 
tasks.  The  integration  of  contractors  with  the  gov¬ 
ernment  on  issues  relative  to  a  given  program  truly 
occurs  at  the  program  IPT  level.  The  development 
teams  (product  and  process  teams)  described  ear¬ 
lier  in  this  chapter  would  be  considered  program 
IPTs.  Program  IPTs  would  also  include  teams 
formed  for  business  reasons,  for  example  teams 
established  to  prepare  Planning,  Programming, 
and  Budgeting  System  (PPBS)  documentation, 
to  prepare  for  Milestone  Approval,  to  develop 
the  RFP,  or  the  like. 
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SUPPLEMENT  B 

GOVERNMENT  ROLE  ON  IPTs 


The  following  list  was  developed  by  the  Ad¬ 
vanced  Amphibious  Assault  Vehicle  (AAAV) 
program  to  inform  its  government  personnel  of 
their  role  on  contractor/government  integrated 
teams.  It  addresses  government  responsibilities 
and  the  realities  imposed  by  contractual  and  le¬ 
gal  constraints.  Though  it  is  specific  to  the  AAAV 
case,  it  can  be  used  as  guidance  in  the  develop¬ 
ment  of  team  planning  for  other  programs. 

1.  The  IPTs  are  contractor-run  entities.  We  do 
not  lead  or  manage  the  IPTs. 

2.  We  serve  as  “customer”  representatives  on  the 
IPTs.  We  are  there  to  REDUCE  THE  CYCLE 
TIME  of  Contractor-Government  (customer) 
communication.  In  other  words,  we  facilitate 
contractor  personnel  getting  Government 
input  faster.  Government  IPT  members  also 
enable  us  to  provide  the  contractor  IPT  Status 
and  issue  information  up  the  Government 
chain  on  a  daily  basis  (instead  of  monthly  or 
quarterly). 

3.  WE  DO  NOT  DO  the  contractor’s  IPT  WORK, 
or  any  portion  of  their  work  or  tasks.  The  con¬ 
tractor  has  been  contracted  to  perform  the  tasks 
outlined  in  the  contract  SOW;  their  personnel 
and  their  subcontractors’  personnel  will  per¬ 
form  those  tasks,  not  us.  But  Government  IPT 
members  will  be  an  active  part  of  the  delib¬ 
erations  during  the  development  of,  and  par¬ 
ticipate  in  “on-the-fly”  reviews  of  deliverables 
called  out  in  CDRLs. 

4.  When  asked  by  contractor  personnel  for  the 
Government’s  position  or  interpretation, 
Government  IPT  members  can  offer  their  per¬ 
sonal  opinion,  as  an  IPT  member,  or  offer 
expert  opinion;  you  can  provide  guidance  as 
to  our  “customer”  opinion  and  what  might 


be  acceptable  to  the  Government  but  you 
can  only  offer  the  “Government”  position 
for  items  that  have  been  agreed  to  by  you 
and  your  Supervisor.  IT  IS  UP  TO  YOUR 
SUPERVISORS  TO  EMPOWER  EACH  OF 
YOU  TO  AN  APPROPRIATE  LEVEL  OF 
AUTHORITY.  It  is  expected  that  this  will 
start  at  a  minimal  level  of  authority  and  be 
expanded  as  each  individual’s  IPT  experi¬ 
ence  and  program  knowledge  grows.  How¬ 
ever...  (see  items  5  &  6). 

5.  Government  IPT  members  CAN  NOT  autho¬ 
rize  any  changes  or  deviations  to/from  the  con¬ 
tract  SOW  or  Specifications.  Government  IPT 
members  can  participate  in  the  deliberations 
and  discussions  that  would  result  in  the  sug¬ 
gestion  of  such  changes.  If/When  an  IPT  con¬ 
cludes  that  the  best  course  of  action  is  not  in 
accordance  with  the  contract,  and  a  contract 
change  is  in  order,  then  the  contractor  must 
submit  a  Contract  Change  Request  (CCR) 
through  normal  channels. 

6.  Government  IPT  members  CAN  NOT  autho¬ 

rize  the  contractor  to  perform  work  that  is  in 
addition  to  the  SOW/contract  requirements. 
The  contractor  IPTs  can  perform  work  that  is 
not  specifically  required  by  the  contract,  at 
their  discretion  (provided  they  stay  within  the 
resources  as  identified  in  the  Team  Operating 
Contract  (TOC). 

7.  Government  IPT  member  participation  in 
contractor  IPT  activities  IS  NOT  Government 
consent  that  the  work  is  approved  by  the 
Government  or  is  chargeable  to  the  contract. 
If  an  IPT  is  doing  something  questionable, 
identify  it  to  your  supervisor  or  Program 
Management  Team  (PMT)  member. 
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8.  Government  members  of  IPTs  do  not  ap¬ 
prove  or  disapprove  of  IPT  decisions,  plans, 
or  reports.  You  offer  your  opinion  in  their 
development,  you  vote  as  a  member,  and  you 
coordinate  issues  with  your  Supervisor  and 
bring  the  “Government”  opinion  (in  the  form 
of  your  opinion)  back  to  the  IPT,  with  the 
goal  of  improving  the  quality  of  the  prod¬ 
ucts;  you  don’t  have  veto  power. 

9.  Government  IPT  members  are  still  subject 
to  all  the  Government  laws  and  regulations 
regarding  “directed  changes,”  ethics,  and 
conduct.  Your  primary  function  is  to  perform 
those  functions  that  are  best  done  by  Gov¬ 
ernment  employees,  such  as: 


•  Conveying  to  contractor  personnel  your 
knowledge/expertise  on  Marine  Corps 
operations  and  maintenance  techniques; 

•  Interfacing  with  all  other  Government 
organizations  (eg.  T&E); 

•  Control/  facilitization  of  government  fur¬ 
nished  equipment  and  materials  (GFE  and 
GFM); 

•  Ensuring  timely  payment  of  submitted 
vouchers; 

•  Full  participation  in  Risk  Management. 
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19.1  INTRODUCTION 

This  chapter  describes  how  the  systems  engineer 
supports  the  development  and  maintenance  of  the 
agreement  between  the  project  office  and  the  con¬ 
tractor  that  will  perform  or  manage  the  detail  work 
to  achieve  the  program  objectives.  This  agreement 
has  to  satisfy  several  stakeholders  and  requires 
coordination  between  responsible  technical,  mana¬ 
gerial,  financial,  contractual,  and  legal  personnel. 
It  requires  a  document  that  conforms  to  the  Fed¬ 
eral  Acquisition  Regulations  (and  supplements), 
program  PPBS  documentation,  and  the  System 
Architecture.  As  shown  by  Figure  19-1,  it  also  has 
to  result  in  a  viable  cooperative  environment  that 
allows  necessary  integrated  teaming  to  take  place. 


The  role  of  technical  managers  or  systems 
engineers  is  crucial  to  satisfying  these  diverse 
concerns.  Their  primary  responsibilities  include: 

•  Supporting  or  initiating  the  planning  effort. 
The  technical  risk  drives  the  schedule  and 
cost  risks  which  in  turn  should  drive  the  type 
of  contractual  approach  chosen, 

•  Prepares  or  supports  the  preparation  of  the 
source  selection  plan  and  solicitation  clauses 
concerning  proposal  requirements  and  selection 
criteria, 

•  Prepares  task  statements, 


Figure  19-1.  Contracting  Process 
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•  Prepares  the  Contract  Data  Requirements  List 
(CDRL), 

•  Supports  negotiation  and  participates  in  source 
selection  evaluations, 

•  Forms  Integrated  Teams  and  coordinates  the 
government  side  of  combined  government  and 
industry  integrated  teams, 

•  Monitors  the  contractor’s  progress,  and 

•  Coordinates  government  action  in  support  of 
the  contracting  officer. 

This  chapter  reflects  the  DoD  approach  to  con¬ 
tracting  for  system  development.  It  assumes  that 
there  is  a  government  program  or  project  office 
that  is  tasking  a  prime  contractor  in  a  competitive 
environment.  However,  in  DoD  there  is  variation 
to  this  theme.  Some  project  activities  are  tasked 
directly  to  a  government  agency  or  facility,  or  are 
contracted  sole  source.  The  processes  described 
in  this  chapter  should  be  tailored  as  appropriate 
for  these  situations. 


19.2  SOLICITATION  DEVELOPMENT 

As  shown  by  Figure  19-2,  the  DoD  contracting 
process  begins  with  planning  efforts.  Planning 
includes  development  of  a  Request  for  Proposal 
(RFP),  specifications,  a  Statement  of  Objective 
(SOO)  or  Statement  of  Work  (SOW),  a  source 
selection  plan,  and  the  Contract  Data  Requirements 
List  (CDRL). 

Request  for  Proposal  (RFP) 

The  RFP  is  the  solicitation  for  proposals.  The  gov¬ 
ernment  distributes  it  to  potential  contractors.  It 
describes  the  government’s  need  and  what  the 
offeror  must  do  to  be  considered  for  the  contract. 
It  establishes  the  basis  for  the  contract  to  follow. 

The  key  systems  engineering  documents  included 
in  a  solicitation  are: 

•  A  statement  of  the  work  to  be  performed.  In 
DoD  this  is  a  Statement  of  Work  (SOW.)  A 
Statement  of  Objectives  (SOO)  can  be  used  to 
obtain  a  SOW  or  equivalent  during  the  selection 
process. 
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•  A  definition  of  the  system.  Appropriate  speci¬ 
fications  and  any  additional  baseline  informa¬ 
tion  necessary  for  clarification  form  this 
documentation.  This  is  generated  by  the 
systems  engineering  process  as  explained 
earlier  in  this  book. 

•  A  definition  of  all  data  required  by  the  cus¬ 
tomer.  In  DoD  this  accomplished  through  use 
of  the  Contract  Data  Requirements  List  (CDRL). 

The  information  required  to  be  in  the  proposals 
responding  to  the  solicitation  is  also  key  for  the 
systems  engineer.  An  engineering  team  will  decide 
the  technical  and  technical  management  merits  of 
the  proposals.  If  the  directions  to  the  offerors  are 
not  clearly  and  correctly  stated,  the  proposal  will 
not  contain  the  information  needed  to  evaluate  the 
offerors.  In  DoD  Sections  L  and  M  of  the  RFP  are 
those  pivotal  documents. 

Task  Statement 

The  task  statement  prepared  for  the  solicitation 
will  govern  what  is  actually  received  by  the 
government,  and  establish  criteria  for  judging 
contractor  performance.  Task  requirements  are 


expressed  in  the  SOW.  During  the  solicitation 
phase  the  tasks  can  be  defined  in  very  general  way 
by  a  SOO.  Specific  details  concerning  SOOs  and 
SOWs  are  attached  at  the  end  of  this  chapter. 

As  shown  by  Figure  19-3,  solicitation  tasking 
approaches  can  be  categorized  into  four  basic 
options:  use  of  a  basic  operational  need,  a  SOO,  a 
SOW,  or  a  detail  specification. 

Option  1  maximizes  contractor  flexibility  by  sub¬ 
mitting  the  Operational  Requirements  Document 
(ORD)  to  offerors  as  a  requirements  document  (e.g. 
in  place  of  SOO/SOW),  and  the  offerors  are 
requested  to  propose  a  method  of  developing  a 
solution  to  the  ORD.  The  government  identifies 
its  areas  of  concern  in  section  M  (evaluation  fac¬ 
tors)  of  the  RFP  to  provide  guidance.  Section  L 
(instructions  to  the  offerors)  should  require  the  bid¬ 
ders  write  a  SOW  based  on  the  ORD  as  part  of 
their  proposal.  The  offeror  proposes  the  type  of 
system.  The  contractor  develops  the  system  speci¬ 
fication  and  the  Work  Breakdown  Structure.  In 
general  this  option  is  appropriate  for  early  efforts 
where  contractor  input  is  necessary  to  expand  the 
understanding  of  physical  solutions  and  alternative 
system  approaches. 
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Figure  19-3.  Optional  Approaches 
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Option  2  provides  moderate  contractor  flexibil¬ 
ity  by  submitting  a  Statement  of  Objectives 
(SOO)  to  the  offerors  as  the  section  C  task  docu¬ 
ment  (e.g.  in  place  of  SOW.)  The  government 
identifies  its  areas  of  concern  in  section  M  (evalu¬ 
ation  factors)  to  provide  guidance.  Section  L  (in¬ 
structions  to  the  offerors)  should  require  as  part 
of  the  proposal  that  offerors  write  a  SOW  based 
on  the  SOO.  In  this  case  the  government  usually 
selects  the  type  of  system,  writes  a  draft  techni¬ 
cal-requirements  document  or  system  specifica¬ 
tion,  and  writes  a  draft  WBS.  This  option  is  most 
appropriate  when  previous  efforts  have  not  de¬ 
fined  the  system  tightly.  The  effort  should  not 
have  any  significant  design  input  from  the  pre¬ 
vious  phase.  This  method  allows  for  innovative 
thinking  by  the  bidders  in  the  proposal  stage.  It 
is  a  preferred  method  for  design  contracts. 

Option  3  lowers  contractor  flexibility,  and  in¬ 
creases  clarity  of  contract  requirements.  In  this 
option  the  Statement  of  Work  (SOW)  is  provided 
to  the  Contractor  as  the  contractual  task  require¬ 
ments  document.  The  government  provides 
instructions  in  section  L  to  the  offerors  to  describe 
the  information  needed  by  the  government  to  evalu¬ 
ate  the  contractor’s  ability  to  accomplish  the  SOW 
tasks.  The  government  identifies  evaluation  fac¬ 
tors  in  section  M  to  provide  guidance  for  priority 
of  the  solicitation  requirements.  In  most  cases,  the 
government  selects  the  type  of  system,  and  pro¬ 
vides  the  draft  system  spec,  as  well  as  the  draft 
WBS.  This  option  is  most  appropriate  when  pre¬ 
vious  efforts  have  defined  the  system  to  the  lower 
WBS  levels  or  where  the  product  baseline  defines 
the  system.  Specifically  when  there  is  substantial 
input  from  the  previous  design  phase  and  there  is 
a  potential  for  a  different  contractor  on  the  new 
task,  the  SOW  method  is  appropriate. 

Option  4  minimizes  contractor  flexibility,  and 
requires  maximum  clarity  and  specificity  of  con¬ 
tract  requirements.  This  option  uses  an  Invitation 
for  Bid  (IFB)  rather  than  an  RFP.  It  provides  bid¬ 
ders  with  specific  detailed  specifications  or  task 
statements  describing  the  contract  deliverables. 
They  tell  the  contractor  exactly  what  is  required 
and  how  to  do  it.  Because  there  is  no  flexibility  in 
the  contractual  task,  the  contract  is  awarded  based 


on  the  low  bid.  This  option  is  appropriate  when 
the  government  has  detailed  specifications  or  other 
product  baseline  documentation  that  defines  the 
deliverable  item  sufficient  for  production.  It  is 
generally  used  for  simple  build-to-print 
reprocurement. 

Data  Requirements 

As  part  of  the  development  of  an  Invitation  for 
Bid  or  Request  for  Proposals,  the  program  office 
typically  issues  a  letter  that  describes  the  planned 
procurement  and  asks  integrated  team  leaders  and 
affected  functional  managers  to  identify  and  jus¬ 
tify  their  data  requirements  for  that  contract.  The 
data  should  be  directly  associated  with  a  process 
or  task  the  contractor  is  required  to  perform. 

The  affected  teams  or  functional  offices  then 
develop  a  description  of  each  data  item  needed. 
Data  Item  Descriptions,  located  in  the  Acquisi¬ 
tion  Management  Systems  &  Data  Requirements 
Control  List  (AMSDL),  can  be  used  for  guidance 
in  developing  these  descriptions.  Descriptions 
should  be  performance  based,  and  format  should 
be  left  to  the  contractor  as  long  as  all  pertinent 
data  is  included.  The  descriptions  are  then 
assembled  and  submitted  for  inclusion  in  the 
solicitation.  The  listing  of  data  requirements  in  the 
contract  follows  an  explicit  format  and  is  referred 
to  as  the  Contract  data  Requirements  List  (CDRL). 

In  some  cases  the  government  will  relegate  the 
data  call  to  the  contractor.  In  this  case  it  is  impor¬ 
tant  that  the  data  call  be  managed  by  a  govern¬ 
ment/  contractor  team,  and  any  disagreements  be 
resolved  prior  to  formal  contract  change  incorpo¬ 
rating  data  requirements.  When  a  SOO  approach 
is  used,  the  contractor  should  be  required  by  section 
L  to  propose  data  requirements  that  correspond  to 
their  proposed  SOW. 

There  is  current  emphasis  on  electronic  submis¬ 
sion  of  contractually  required  data.  Electronic  Data 
Interchange  (EDI)  sets  the  standards  for  compatible 
data  communication  formats. 

Additional  information  on  data  management,  types 
of  data,  contractual  considerations,  and  sources  of 


174 


Chapter  19 


Contractual  Considerations 


data  are  presented  in  chapters  10  and  13.  Addi¬ 
tional  information  on  CDRLs  is  provided  at  the 
end  of  this  chapter. 

Technical  Data  Package  Controversy 

Maintenance  of  a  detailed  baseline  such  as  the  “as 
built”  description  of  the  system,  usually  referred 
to  as  a  Technical  Data  Package  (TDP),  can  be  very 
expensive  and  labor  intensive.  Because  of  this, 
some  acquisition  programs  may  not  elect  to  pur¬ 
chase  this  product  description.  If  the  Government 
will  not  own  the  Technical  Data  Package  the 
following  questions  must  be  resolved  prior  to 
solicitation  issue: 

•  What  are  the  pros  and  cons  associated  with  the 
TDP  owned  by  the  contractor? 

•  What  are  the  support  and  reprocurement 
impacts? 

•  What  are  the  product  improvement  impacts? 

•  What  are  the  open  system  impacts? 

In  general  the  government  should  have  sufficient 
data  rights  to  address  life  cycle  concerns,  such  as 
maintenance  and  product  upgrade.  The  extent  to 
which  government  control  of  configurations  and 
data  is  necessary  will  depend  on  support  and 
reprocurement  strategies.  This,  in  turn,  demands 
that  those  strategic  decisions  be  made  as  early  as 
possible  in  the  system  development  to  avoid 
purchasing  data  rights  as  a  hedge  against  the 
possibility  that  the  data  will  be  required  later  in 
the  program  life  cycle. 

Source  Selection 

Source  Selection  determines  which  offeror  will  be 
the  contractor,  so  this  choice  can  have  profound 
impact  on  program  risk.  The  systems  engineer 
must  approach  the  source  selection  with  great  care 
because,  unlike  many  planning  decisions  made 
early  in  product  life  cycles,  the  decisions  made 
relative  to  source  selection  can  generally  not  be 
easily  changed  once  the  process  begins.  Laws  and 
regulations  governing  the  fairness  of  the  process 


require  that  changes  be  made  very  carefully — and 
often  at  the  expense  of  considerable  time  and  effort 
on  the  part  of  program  office  and  contractor  per¬ 
sonnel.  In  this  environment,  even  minor  mistakes 
can  cause  distortion  of  proper  selection. 

The  process  starts  with  the  development  of  a 
Source  Selection  Plan  (SSP),  that  relates  the 
organizational  and  management  structure,  the 
evaluation  factors,  and  the  method  of  analyzing 
the  offerors’  responses.  The  evaluation  factors  and 
their  priority  are  transformed  into  information  pro¬ 
vided  to  the  offerors  in  sections  L  and  M  of  the 
RFP.  The  offerors’  proposals  are  then  evaluated 
with  the  procedures  delineated  in  the  SSP.  These 
evaluations  establish  which  offerors  are  conform¬ 
ing,  guide  negotiations,  and  are  the  major  factor 
in  contractor  selection.  The  SSP  is  further 
described  at  the  end  of  this  chapter. 

The  system  engineering  area  of  responsibility 
includes  support  of  SSP  development  by: 

•  Preparing  the  technical  and  technical  manage¬ 
ment  parts  of  evaluation  factors, 

•  Organizing  technical  evaluation  team(s),  and 

•  Developing  methods  to  evaluate  offerors’  pro¬ 
posals  (technical  and  technical  management). 

19.3  SUMMARY  COMMENTS 

•  Solicitation  process  planning  includes  devel¬ 
opment  of  a  Request  for  Proposal,  specifica¬ 
tions,  a  Statement  of  Objective  or  Statement  of 
Work,  a  source  selection  plan,  and  the  Contract 
Data  Requirements  List. 

•  There  are  various  options  available  to  program 
offices  as  far  as  the  guidance  and  constraints 
imposed  on  contractor  flexibility.  The  govern¬ 
ment,  in  general,  prefers  that  solicitations  be 
performance-based . 

•  Data  the  contractor  is  required  to  provide  the 
government  is  listed  on  the  Contract  Data 
Requirements  List. 
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•  Source  Selection  is  based  on  the  evaluation  cri-  reflected  in  Sections  L  &  M  of  the  Request  for 
teria  outlined  in  the  Source  Selection  Plan  and  Proposal  (RFP). 
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SUPPLEMENT  A 

STATEMENT  OF  OBJECTIVES 

(SOO) 


The  SOO  is  an  alternative  to  a  government  pre¬ 
pared  SOW.  A  SOO  provides  the  Government’s 
overall  objectives  and  the  offeror’s  required  sup¬ 
port  to  achieve  the  contractual  objectives.  Offerors 
use  the  SOO  as  a  basis  for  preparing  a  SOW  which 
is  then  included  as  an  integral  part  of  the  proposal 
which  the  government  evaluates  during  the  source 
selection. 

Purpose 

SOO  expresses  the  basic,  top-level  objectives  of 
the  acquisition  and  is  provided  in  the  RFP  in  lieu 
of  a  government-written  SOW.  This  approach  gives 
the  offerors  the  flexibility  to  develop  cost  effec¬ 
tive  solutions  and  the  opportunity  to  propose 
innovative  alternatives. 

Approach 

The  government  includes  a  brief  (1-  to  2-page) 
SOO  in  the  RFP  and  requests  that  offerors  pro¬ 
vide  a  SOW  in  their  proposal.  The  SOO  is  typi¬ 
cally  appended  to  section  J  of  the  RFP  and  does 
not  become  part  of  the  contract.  Instructions  for 
the  contractor  prepared  SOW  would  normally  be 
included  in  or  referenced  by  section  L. 

SOO  Development 

Step  1:  The  RFP  team  develops  a  set  of  objectives 
compatible  with  the  overall  program  direction 
including  the  following: 

•  User(s)  operational  requirements, 

•  Programmatic  direction, 


•  Draft  technical  requirements,  and 

•  Draft  WBS  and  dictionary. 

Step  2:  Once  the  program  objectives  are  defined, 
the  SOO  is  constructed  so  that  it  addresses  prod¬ 
uct-oriented  goals  and  performance-oriented 
requirements. 

SOO  and  Proposal  Evaluations 

Section  L  (Instructions  to  Offerors)  of  the  RFP 
must  include  instructions  to  the  offeror  that  require 
using  the  SOO  to  construct  and  submit  a  SOW.  In 
Section  M  (Evaluation  Criteria)  the  program  office 
should  include  the  criteria  by  which  the  proposals, 
including  the  contractor’s  draft  SOW,  will  be  evalu¬ 
ated.  Because  of  its  importance,  the  government’s 
intention  to  evaluate  the  proposed  SOW  should 
be  stressed  in  Sections  L  and  M. 

Offeror  Development  of 
the  Statement  of  Work 

The  offeror  should  establish  and  define  in  clear, 
understandable  terms: 

•  Non-specification  requirements  (the  tasks  that 
the  contractor  must  do), 

•  What  has  to  be  delivered  or  provided  in  order 
for  him  to  get  paid, 

•  What  data  is  necessary  to  support  the  effort,  and 

•  Information  that  would  show  how  the  offerors 
would  perform  the  work  that  could  differen¬ 
tiate  between  them  in  proposal  evaluation  and 
contractor  selection. 
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At  contract  award  the  SOW,  as  changed  through  the  standard  for  measuring  contractor’s  effective- 
negotiations,  becomes  part  of  the  contract  and  ness. 


SOO  Example: 

Joint  Air-to-Surface  Standoff  Missile  (JASSM) 

Statement  of  Objectives 

The  Air  Force  and  Navy  warfighters  need  a  standoff  missile  that  will  destroy  the  enemies’  war- 

sustaining  capabilities  with  a  launch  standoff  range  outside  the  range  of  enemy  area  defenses. 

Offerors  shall  use  the  following  objectives  for  the  pre-EMD  and  EMD  acquisition  phases  of  the 

JASSM  program  along  with  other  applicable  portions  of  the  RFP  when  preparing  proposals  and 

program  plans.  IMP  events  shall  be  traceable  to  this  statement  of  objectives: 

Pre-EMD  Objectives 

a.  Demonstrate,  at  the  sub-system  level  as  a  minimum,  end-to-end  performance  of  the  system 
concept.  Performance  will  be  at  the  contractor-developed  System  Performance  Specifica¬ 
tion  requirements  level  determined  during  this  phase  without  violation  of  any  key  performance 
parameters. 

b.  Demonstrate  the  ability  to  deliver  an  affordable  and  producible  system  at  or  under  the  average 
unit  procurement  price  (AUPP). 

c.  Provide  a  JASSM  system  review  including  final  system  design,  technical  accomplishments, 
remaining  technical  risks  and  major  tasks  to  be  accomplished  in  EMD. 

EMD  Objectives 

a.  Demonstrate  through  test  and/or  analysis  that  all  requirements  as  stated  in  the  contractor 
generated  System  Performance  Specification,  derived  from  Operational  Requirements,  are 
met,  including  military  utility  (operational  effectiveness  and  suitability). 

b.  Demonstrate  ability  to  deliver  an  affordable  and  producible  system  at  or  under  the  AUPP 
requirement. 

c.  Demonstrate  all  production  processes. 

d.  Produce  production  representative  systems  for  operational  test  and  evaluation,  including 
combined  development  /  operational  test  and  evaluation. 
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SUPPLEMENT  B 

STATEMENT  OF  WORK 
(SOW) 


The  SOW  is  a  specific  statement  of  the  work  to  be 
performed  by  the  contractor.  It  is  derived  from  the 
Program  WBS  (System  Architecture).  It  should 
contain,  at  a  minimum,  a  statement  of  scope  and 
intent,  as  well  as  a  logical  and  clear  definition  of 
all  tasks  required.  The  SOW  normally  consists  of 
three  parts: 

Section  1:  Scope  -  Defines  overall  purpose  of 
the  program  and  to  what  the  SOW  applies. 

Section  2:  Applicable  Documents  -  Lists  the 
specifications  and  standards  referenced  in  Sec¬ 
tion  3. 

Section  3:  Requirements  -  States  the  tasks  the 
contractor  has  to  perform  to  provide  the 


deliverables.  Tasks  should  track  with  the  WBS. 
The  SOW  describes  tasks  the  contractor  has  to 
do.  The  specifications  describe  the  products. 

Statement  of  Work  Preparation 
and  Evaluation  Strategies 

SOWs  should  be  written  by  an  integrated  team 
of  competent  and  experienced  members.  The 
team  should: 

•  Review  and  use  the  appropriate  WBS  for  the 
SOW  framework, 

•  Set  SOW  objectives  in  accordance  with  the 
Acquisition  Plan  and  systems  engineering 
planning, 


Figure  19-4.  Requirement-WBS-SOW  Flow 
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•  Develop  a  SOW  tasking  outline  and  check¬ 
list, 

•  Establish  schedule  and  deadlines,  and 

•  Develop  a  comprehensive  SOW  from  the 
above. 

Performance-based  SOW 

The  term  performance-based.  SOW  has  become 
a  common  expression  that  relates  to  a  SOW  that 
tasks  the  contractor  to  perform  the  duties  neces¬ 
sary  to  provide  the  required  deliverables,  but  is 
not  specific  as  to  the  process  details.  Basically,  all 
SOWs  should  be  performance  based,  however,  past 
DoD  generated  SOWs  have  had  the  reputation  of 
being  overly  directive.  A  properly  developed  SOW 
tasks  the  contractor  without  telling  him  how  to 
accomplish  the  task. 

Evaluating  the  SOW 

The  WBS  facilitates  a  logical  arrangement  of  the 
elements  of  the  SOW  and  a  tracing  of  work  effort 
expended  under  each  of  the  WBS  elements.  It  helps 
integrated  teams  to  ensure  all  requirements  have 
been  included,  and  provides  a  foundation  for  track¬ 
ing  program  evolution  and  controlling  the  change 
process.  As  shown  by  Figure  19-4,  the  WBS  serves 
as  a  link  between  the  requirements  and  the  SOW. 

In  the  past,  DoD  usually  wrote  the  SOW  and,  over 
time,  an  informal  set  of  rules  had  been  developed 
to  assist  in  drafting  them.  While  the  government 
today  generally  does  not  write  the  SOW,  but,  rather, 
more  often  evaluates  the  contractor’s  proposed 
statement  of  work,  those  same  rules  can  assist  in 
the  government  role  of  evaluator. 

Statement  of  Work  Rules 

In  section  1.  Scope : 

DO  NOT: 

•  Include  directed  work  statements. 

•  Include  data  requirements  or  deliverable 
products. 


In  section  2.  Applicable  Documents: 

DO  NOT: 

•  Include  guidance  documents  that  apply  only  to 
Government  PMOs  (e.g.  DoD  5000  series  and 
service  regulations). 

In  section  3.  Requirements: 

DO  NOT: 

•  Define  work  tasks  in  terms  of  data  to  be 
delivered. 

•  Order,  describe,  or  discuss  CDRL  data  (OK  to 
reference). 

•  Express  work  tasks  in  data  terms. 

•  Invoke,  cite,  or  discuss  a  DID. 

•  Invoke  handbooks,  service  regulations, 
technical  orders,  or  any  other  document  not 
specifically  written  in  accordance  with  MIL- 
STD-96 1/962. 

•  Specify  how  task  is  to  be  accomplished. 

•  Use  the  SOW  to  amend  contract  specifications. 

•  Specify  technical  proposal  or  performance 
criteria  or  evaluation  factors. 

•  Establish  delivery  schedules. 

•  Over  specify. 

In  section  3.  Requirements: 

DO: 

•  Specify  work  requirements  to  be  performed 
under  contract. 

•  Set  SOW  objectives  to  reflect  the  acquisition 
plan  and  systems  engineering  planning. 

•  Provide  a  priceable  set  of  tasks. 
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Express  work  to  be  accomplished  in  work 
words. 

Use  “shall”  whenever  a  task  is  mandatory. 

Use  “will”  only  to  express  a  declaration  of 
purpose  or  simple  futurity. 


•  Use  WBS  as  an  outline. 

•  List  tasks  in  chronological  order. 

•  Limit  paragraph  numbering  to  3rd  sub-level 
(3.3.1. 1.)  -  Protect  Government  interests. 

•  Allow  for  contractor’s  creative  effort. 
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SUPPLEMENT  C 

CONTRACT  DATA 
REQUIREMENTS  LIST 


The  Contract  Data  Requirements  List  (CDRL) 
is  a  list  of  authorized  data  requirements  for  a 
specific  procurement  that  forms  a  part  of  the 
contract.  It  is  comprised  of  a  series  of  DD  Forms 
1423  (Individual  CDRL  forms)  containing  data 
requirements  and  delivery  instructions.  CDRLs 
should  be  linked  directly  to  SOW  tasks  and  man¬ 


aged  by  the  program  office  data  manager.  A 
sample  CDRL  data  requirement  is  shown  in  Fig¬ 
ure  19-5. 

Data  requirements  can  also  be  identified  in  the 
contract  via  Special  Contract  Clauses  (Federal 
Acquisition  Clauses.)  Data  required  by  the  FAR 
clauses  are  usually  required  and  managed  by  the 
Contracting  Officer. 


CONTRACT  DATA  REQUIREMENTS  LIST 


ATCH  NR:  3  TO  EXHIBIT: 

TO  CONTRACT/PR:  F33657-86-C-2085 


CATEGORY:  X 


SYSTEM/ITEM:  ATF  DEM/VAL  PHASE 
CONTRACTOR:  LOCKHEED 


1) 

2)  SOW  3.1 

6) 

10) 

12) 

14) 

3100 

3) 

ASD/TASE 

ONE/R 

60DAC 

ASD/TASE 

2/0 

4) 

5)  SOW  3.1 

7) 

8) 

9) 

OTE62011 

IT 

D 

13) 

SEE  16 


16) 

BLK4: 


BLK  4:  SEE  APPENDIXES  TO  CDRL  FOR  DID. 

THIS  DID  IS  TAILORED  AS  FOLLOWS: 

(1)  CONTRACTOR  FORMAT  IS  ACCEPTABLE. 

(2)  CHANGE  PARAGRAPH  2a  OF  DID  TO  READ:  “PROGRAM  RISK 
ANALYSIS. THIS  SECTION  SHALL  DESCRIBE  THE  PLAN  AND 
METHODOLOGY  FOR  A  CONTINUING  ASSESSMENT  OF 
TECHNICAL,  SUPPORTABILITY,  COST,  AND  SCHEDULE  RISKS  OF 
THE  SYSTEM  PROGRAM. THIS  SECTION  SHOULD  BE 
CONSISTENT  WITH  AND  NOT  DUPLICATE  THE  SYSTEM 
INTEGRATION  PLAN  (REFERENCE  DI-S-3563/T);  I.E.,  ONE  PLAN 
MAY  REFERENCE  THE  OTHER.” 

BLK  13:  REVISIONS  SHALL  BE  SUBMITTED  AS  REQUIRED  BY  CHANGE 
RESULTING  FROM  THE  SYSTEMS  ENGINEERING  PROCESS. 
NOTE:  SCHEDULES  ASSOCIATED  WITH  THIS  PLAN  SHALL  BE 
INTEGRATED  WITH  THE  MASTER  PROGRAM  PLANNING 
SCHEDULE  SUBMITTED  ON  MAGNETIC  MEDIA  IN  ACCORDANCE 
WITH  DI-A-3007/T. 


PREPARED  BY: 

DATE: 

86JUN11 

APPROVED  BY: 

DATE: 

86  JUNE  11 

DD  FORM  1423  ADPE  ADAPTATION  SEP  81  (ASD/YYD) 

Figure  19-5.  CDRL  Single  Data  Item  Requirement  Example 
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Data  Requirement  Sources 

Standard  Data  Item  Descriptions  (DID)  define  data 
content,  preparation  instructions,  format,  intended 
use,  and  recommended  distribution  of  data  required 
of  the  contractor  for  delivery.  The  Acquisition 
Management  Systems  and  Data  Requirements 
Control  List  (AMSDL)  identifies  acquisition  man¬ 
agement  systems,  source  documents,  and  standard 
Data  Item  Descriptions  (DIDs).  With  acquisi¬ 
tion  reform  the  use  of  DIDs  has  declined,  and 
data  item  requirements  now  are  either  tailored 
DIDs  or  a  set  of  requirements  specifically  writ¬ 
ten  for  the  particular  RFP  in  formats  agreeable 
to  the  contractor  and  the  government. 

DD  Form  1423  Road  Map 

Block  1 :  Data  Item  Number  -  represents  the  CDRL 
sequence  number. 

Block  2:  Title  of  Data  Item  -  same  as  the  title 
entered  in  item  1  of  the  DID  (DD  Form  1664). 

Block  4:  Authority  (Data  Acquisition  Document 
Number)  -  same  as  item  2  of  the  DID  form  and 
will  include  a  “/t”  to  indicate  DID  has  been  tailored. 

Block  5:  Contract  Reference  -  identifies  the  DID 
authorized  in  block  4  and  the  applicable  document 
and  paragraph  numbers  in  the  SOW  from  which 
the  data  flows. 

Block  6:  Requiring  Office  -  activity  responsible 
for  advising  the  technical  adequacy  of  the  data. 

Block  7:  Specific  Requirements  -  may  be  needed 
for  inspection/acceptance  of  data. 


Block  8:  Approval  Code  -  if  “A,”  it  is  a  critical 
data  item  requiring  specific,  advanced,  written 
approval  prior  to  distribution  of  the  final  data  item. 

Block  9:  Distribution  Statement  Required: 

Category  A  is  unlimited-release  to  the  public. 

Category  B  is  limited-release  to  government 
agencies. 

Category  C  limits  release  to  government  agencies 
and  their  contractors. 

Category  D  is  limited-release  to  DoD  offices  and 
their  contractors. 

Category  E  is  for  release  to  DoD  components  only. 

Category  F  is  released  only  as  directed  and 
normally  classified. 

Block  12:  Date  of  First  Submission  -  indicates 
year/month/day  of  first  submission  and  identifies 
specific  event  or  milestone  data  is  required. 

Block  13:  Date  of  Subsequent  Submission  -  if  data 
is  submitted  more  than  once,  subsequent  dates  will 
be  identified. 

Block  14:  Distribution  -  identify  each  addressee 
and  identify  the  number  of  copies  to  be  received 
by  each.  Use  office  symbols,  format  of  data  to  be 
delivered,  command  initials,  etc. 

Block  1 6:  Remarks  -  explain  only  tailored  features 
of  the  DID,  any  additional  information  for  blocks 
1-15,  and  any  resubmittal  schedule  or  special 
conditions  for  updating  data  submitted  for  gov¬ 
ernment  approval. 
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Figure  19-6.  Source  Selection  Process 
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SSP  Evaluation  Factors 

The  evaluation  factors  are  a  list,  in  order  of  rela¬ 
tive  importance,  of  those  aspects  of  a  proposal  that 
will  be  evaluated  quantitatively  and  qualitatively 
to  arrive  at  an  integrated  assessment  as  to  which 
proposal  can  best  meet  the  Government’s  need  as 
described  in  the  solicitation.  Figure  19-7  shows 
an  example  of  one  evaluation  category,  life  cycle 
cost.  The  purpose  of  the  SSP  evaluation  is  to 
inform  offerors  of  the  importance  the  Government 
attaches  to  various  aspects  of  a  proposal  and  to 
allow  the  government  to  make  fair  and  reasoned 
differentiation  between  proposals. 

In  general  the  following  guidance  should  be  used 
in  preparing  evaluation  factors: 

•  Limit  the  number  of  evaluation  factors, 

•  Tailor  the  evaluation  factors  to  the  Government 
requirement  (e.g.  combined  message  of  the 
SOO/SOW,  specification,  CDRL,  etc.),  and 

•  Cost  is  always  an  evaluation  factor.  The  identi¬ 
fication  of  the  cost  that  is  to  be  used  and  its 
relative  importance  in  rating  the  proposal 
should  be  clearly  identified. 


Factors  to  Consider 

There  is  not  sufficient  space  here  to  attempt  to 
exhaustively  list  all  the  factors  that  might  influence 
the  decision  made  in  a  source  selection.  The 
following  are  indicative  of  some  of  the  key 
consideration,  however: 

•  Is  the  supplier’s  proposal  responsive  to  the 
government’s  needs  as  specified  in  the  request 
for  proposal? 

•  Is  the  supplier’s  proposal  directly  supportive 
of  the  system  requirements  specified  in  the 
system  specification  and  SOO/SOW? 

•  Have  the  performance  characteristics  been 
adequately  specified  for  the  items  proposed? 
Are  they  meaningful,  measurable,  and  traceable 
from  the  system-level  requirements? 

•  Have  effectiveness  factors  been  specified 
(e.g.  reliability,  maintainability,  supportabil- 
ity,  and  availability?)  Are  they  meaningful, 
measurable,  and  traceable,  from  the  system- 
level  requirements? 

•  Has  the  supplier  addressed  the  requirement 
for  test  and  evaluation  of  the  proposed  sys¬ 
tem  element? 


Rating 

(Points) 

Evaluation  Criteria  -  Life  Cycle  Cost 

9-10 

Offeror  has  included  a  complete  Life  Cycle  Cost  analysis  that  supports  his  proposal. 

7-8 

Offeror  did  not  include  a  complete  Life  Cycle  Cost  analysis  but  has  supported  his 
design  approach  on  the  basis  of  Life  Cycle  Cost. 

5-6 

Offeror  plans  to  complete  a  Life  Cycle  Cost  analysis  as  part  of  the  contract  effort  and 
has  described  the  process  that  will  be  used. 

3-4 

Offeror  plans  to  complete  a  Life  Cycle  Cost  analysis  as  part  of  the  contract  effort  but 
did  not  describe  the  process  that  will  be  used. 

0-2 

Life  Cycle  Cost  was  not  addressed  in  the  Offeror’s  proposal. 

Figure  19-7.  Evaluation  Factors  Example 
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•  Have  life  cycle  support  requirements  been  iden¬ 
tified  (e.g.  maintenance  resource  requirements, 
spare/repair  parts,  test  and  support  equipment, 
personnel  quantities  and  skills,  etc?)  Have  these 
requirements  been  minimized  to  the  extent 
possible  through  design? 

•  Does  the  proposed  design  configuration  re¬ 
flect  growth  potential  or  change  flexibility? 

•  Has  the  supplier  developed  a  comprehensive 
manufacturing  and  construction  plan?  Are  key 
manufacturing  processes  identified  along  with 
their  characteristics? 

•  Does  the  supplier  have  an  adequate  quality 
assurance  and  statistical  process  control 
programs? 

•  Does  the  supplier  have  a  comprehensive 
planning  effort  (e.g.  addresses  program  tasks, 
organizational  structure  and  responsibilities,  a 
WBS,  task  schedules,  program  monitoring  and 
control  procedures,  etc.)? 


•  Does  the  supplier’s  proposal  address  all  aspects 
of  total  life  cycle  cost? 

•  Does  the  supplier  have  previous  experience  in 
the  design,  development,  and  production  of  sys¬ 
tem  elements/components  which  are  similar  in 
nature  to  the  item  proposed? 

Proposal  Evaluation 

Proposal  evaluation  factors  can  be  analyzed  with 
any  reasonable  trade  study  approach.  Figure  19-8 
shows  a  common  approach.  In  this  approach  each 
factor  is  rated  based  on  the  evaluation  factor  matrix 
established  for  each  criteria,  such  as  that  shown  in 
Figure  19-7.  It  is  then  multiplied  by  a  weighting 
factor  based  on  the  perceived  priority  of  each 
criteria.  All  the  weighted  evaluations  are  added 
together  and  the  highest  score  wins. 

Like  trade  studies  the  process  should  be  examined 
for  sensitivity  problems;  however,  in  the  case  of 
source  selection,  the  check  must  be  done  with 
anticipated  values  prior  to  release  of  the  RFP. 
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Evaluation  Criteria 

G3 

Proposal  A 

Proposal  B 

Proposal  C 

Rating 

Score 

Rating 

Score 

Rating 

Score 

A.  Technical  Requirements: 

25 

1.  Performance  Characteristics 

6 

4 

24 

5 

30 

5 

2.  Effectiveness  Factors 

4 

3 

12 

4 

16 

3 

3.  Design  Approach 

3 

2 

6 

3 

9 

1 

4.  Design  Documentation 

4 

3 

12 

wm 

16 

8 

5.  Test  and  Evaluation  Approach 

2 

2 

4 

2 

4 

6.  Product  Support  Requirements 

4 

2 

8 

n 

12 

8 

B.  Production  Capability 

20 

m 

1.  Production  Layout 

8 

5 

40 

6 

1m| 

6 

48 

2.  Manufacturing  Process 

5 

2 

10 

3 

4 

20 

3.  Quality  Control  Assurance 

7 

5 

35 

6 

42 

4 

28 

C.  Management 

20 

1.  Planning  (Plans/Schedules) 

6 

4 

24 

5 

30 

4 

24 

2.  Organization  Structure 

4 

4 

16 

4 

12 

4 

16 

3.  Available  Personnel  Resources 

5 

3 

15 

3 

20 

3 

15 

4.  Management  Controls 

5 

3 

15 

3 

20 

4 

20 

D.  Total  Cost 

25 

■ 

■ 

1.  Acquisition  Price 

10 

7 

70 

5 

o 

6 

m 

2.  Life  Cycle  Cost 

15 

9 

135 

10 

m 

8 

m 

E.  Additional  Factors 

10 

■ 

1.  Prior  Experience 

4 

4 

16 

3 

12 

3 

2.  Past  Performance 

6 

5 

30 

5 

30 

3 

n 

Grand  Total 

100 

476 

516* 

450 

*  Select  Proposal  B 

Figure  19-8.  Source  Evaluation 
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MANAGEMENT  CONSIDERATIONS 
AND  SUMMARY 


20.1  MANAGEMENT  CONSIDERATIONS 

The  Acquisition  Reform  Environment 

No  one  involved  in  systems  acquisition,  either 
within  the  department  or  as  a  supplier,  can  avoid 
considering  how  to  manage  acquisition  in  the 
current  reform  environment.  In  many  ways,  re¬ 
thinking  the  way  we  manage  the  systems  engi¬ 
neering  process  is  implicit  in  reforming  acquisi¬ 
tion  management.  Using  performance  specifica¬ 
tions  (instead  of  detailed  design  specifications), 
leaving  design  decisions  in  the  hands  of 
contractors,  delaying  government  control  of 
configuration  baselines — all  are  reform  measures 
related  directly  to  systems  engineering  manage¬ 
ment.  This  text  has  already  addressed  and  ac¬ 
knowledged  managing  the  technical  effort  in  a 
reform  environment. 

To  a  significant  extent,  the  systems  engineering 
processes — and  systems  engineers  in  general — 
are  victims  of  their  own  successes  in  this  envi¬ 
ronment.  The  systems  engineering  process  was 
created  and  evolved  to  bring  discipline  to  the 
business  of  producing  very  complex  systems.  It 
is  intended  to  ensure  that  requirements  are  care¬ 
fully  analyzed,  and  that  they  flow  down  to  de¬ 
tailed  designs.  The  process  demands  that  details 
are  understood  and  managed.  And  the  process 
has  been  successful.  Since  the  1960s  manufac¬ 
turers,  in  concert  with  government  program  of¬ 
fices,  have  produced  a  series  of  ever-increas- 
ingly  capable  and  reliable  systems  using  the 
processes  described  in  this  text.  The  problem 
is,  in  too  many  cases,  we  have  overlaid  the 
process  with  ever-increasing  levels  of  controls, 
reports,  and  reviews.  The  result  is  that  the 
cycle  time  required  to  produce  systems  has  in¬ 


creased  to  unacceptable  levels,  even  as 
technology  life  cycles  have  decreased  precipi¬ 
tously.  The  fact  is  that,  in  too  many  cases,  we 
are  producing  excellent  systems,  but  systems  that 
take  too  long  to  produce,  cost  too  much,  and  are 
often  outdated  when  they  are  finally  produced. 
The  demand  for  change  has  been  sounded,  and 
systems  engineering  management  must  respond 
if  change  is  to  take  place.  The  question  then 
becomes  how  should  one  manage  to  be  success¬ 
ful  in  this  environment?  We  have  a  process  that 
produces  good  systems;  how  should  we  change 
the  process  that  has  served  us  well  so  that  it 
serves  us  better? 

At  the  heart  of  acquisition  reform  is  this  idea: 
we  can  improve  our  ability  to  provide  our  users 
with  highly  capable  systems  at  reasonable  cost 
and  schedule.  We  can  if  we  manage  design  and 
development  in  a  way  that  takes  full  advantage 
of  the  expertise  resident  both  with  the  govern¬ 
ment  and  the  contractor.  This  translates  into  the 
government  stating  its  needs  in  terms  of  perfor¬ 
mance  outcomes  desired,  rather  than  in  terms  of 
specific  design  solutions  required;  and,  likewise, 
in  having  contractors  select  detailed  design  ap¬ 
proaches  that  deliver  the  performance  demanded, 
and  then  taking  responsibility  for  the  performance 
actually  achieved. 

This  approach  has  been  implemented  in  DoD, 
and  in  other  government  agencies  as  well.  In  its 
earlier  implementations,  several  cases  occurred 
where  the  government  managers,  in  an  attempt 
to  ensure  that  the  government  did  not  impose 
design  solutions  on  contractors,  chose  to  delib¬ 
erately  distance  the  government  technical  staff 
from  contractors.  This  presumed  that  the  con¬ 
tractor  would  step  forward  to  ensure  that  neces- 


189 


Systems  Engineering  Fundamentals 


Chapter  20 


sary  engineering  disciplines  and  functions  were 
covered.  In  more  than  one  case,  the  evidence 
after  the  fact  was  that,  as  the  government  stepped 
back  to  a  less  directive  role  in  design  and  devel¬ 
opment,  the  contractor  did  not  take  a  correspond¬ 
ing  step  forward  to  ensure  that  normal  engineer¬ 
ing  management  disciplines  were  included.  In 
several  cases  where  problems  arose,  after-the- 
fact  investigation  showed  important  elements  of 
the  systems  engineering  process  were  either  de¬ 
liberately  ignored  or  overlooked. 

The  problem  in  each  case  seems  to  have  been 
failure  to  communicate  expectations  between  the 
government  and  the  contractor,  compounded  by 
a  failure  on  the  part  of  the  government  to  ensure 
that  normal  engineering  management  disciplines 
were  exercised.  One  of  the  more  important 
lessons  learned  has  been  that  while  the  systems 
engineering  process  can — and  should  be — tai¬ 
lored  to  the  specific  needs  of  the  program,  there 
is  substantial  risk  ignoring  elements  of  the  pro¬ 
cess.  Before  one  decides  to  skip  phases,  elimi¬ 
nate  reviews,  or  take  other  actions  that  appear  to 
deliver  shortened  schedules  and  less  cost,  one 
must  ensure  that  those  decisions  are  appropri¬ 
ate  for  the  risks  that  characterize  the  program. 

Arbitrary  engineering  management  decisions 
yield  poor  technical  results.  One  of  the  primary 
requirements  inherent  in  systems  engineering  is 
to  assess  the  engineering  management  program 
for  its  consistency  with  the  technical  realities 
and  risks  confronted,  and  to  communicate  his/ 
her  findings  and  recommendations  to  manage¬ 
ment.  DoD  policy  is  quite  clear  on  this  issue. 
The  government  is  not,  in  most  cases,  expected 
to  take  the  lead  in  the  development  of  design 
solutions.  That,  however,  does  not  relieve  the 
government  of  its  responsibilities  to  the  taxpay¬ 
ers  to  ensure  that  sound  technical  and  manage¬ 
ment  processes  are  in  place.  The  systems  engi¬ 
neer  must  take  the  lead  role  in  establishing  the 
technical  management  requirements  for  the  pro¬ 
gram  and  seeing  that  those  requirements  are  com¬ 
municated  clearly  to  program  managers  and  to 
the  contractor. 


Communication  -  Trust  and  Integrity 

Clearly,  one  of  the  fundamental  requirements  for 
an  effective  systems  engineer  is  the  ability  to 
communicate.  Key  to  effective  communication 
is  the  rudimentary  understanding  that  communi¬ 
cation  involves  two  elements — a  transmitter  and 
a  receiver.  Even  if  we  have  a  valid  message  and 
the  capacity  for  expressing  our  positions  in  terms 
that  enable  others  to  understand  what  we  are 
saying,  true  communication  may  not  take  place 
if  the  intended  receiver  chooses  not  to  receive 
our  message.  What  can  we  do,  as  engineering 
managers  to  help  our  own  cause  as  far  as  ensur¬ 
ing  that  our  communications  are  received  and 
understood? 

Much  can  be  done  to  condition  others  to  listen 
and  give  serious  consideration  to  what  one  says, 
and,  of  course,  the  opposite  is  equally  true — 
one  can  condition  others  to  ignore  what  he/she 
says.  It  is  primarily  a  matter  of  establishing  cred¬ 
ibility  based  on  integrity  and  trust. 

First,  however,  it  is  appropriate  to  discuss  the 
systems  engineer’s  role  as  a  member  of  the 
management  team.  Systems  engineering,  as 
practiced  in  DoD,  is  fundamentally  the  practice 
of  engineering  management.  The  systems  engi¬ 
neer  is  expected  to  integrate  not  only  the  techni¬ 
cal  disciplines  in  reaching  recommendations,  but 
also  to  integrate  traditional  management  con¬ 
cerns  such  as  cost,  schedule,  and  policy  into  the 
technical  management  equation.  In  this  role,  se¬ 
nior  levels  of  management  expect  the  systems 
engineer  to  understand  the  policies  that  govern 
the  program,  and  to  appreciate  the  imperatives 
of  cost  and  schedule.  Furthermore,  in  the  ab¬ 
sence  of  compelling  reasons  to  the  contrary,  they 
expect  support  of  the  policies  enunciated  and 
they  expect  the  senior  engineer  to  balance  tech¬ 
nical  performance  objectives  with  cost  and 
schedule  constraints. 

Does  this  mean  that  the  engineer  should  place 
his  obligation  to  be  a  supportive  team  member 
above  his  ethical  obligation  to  provide  honest 
engineering  judgment?  Absolutely  not!  But  it 
does  mean  that,  if  one  is  to  gain  a  fair  hearing 
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for  expression  of  reservations  based  on  engineer¬ 
ing  judgment,  one  must  be  viewed  as  a  member 
of  the  team.  The  individual  who  always  fights 
the  system,  always  objects  to  established  policy, 
and,  in  general,  refuses  to  try  to  see  other  points 
of  view  will  eventually  become  isolated.  When 
others  cease  listening,  the  communication  stops 
and  even  valid  points  of  view  are  lost  because 
the  intended  audience  is  no  longer  receiving  the 
message — valid  or  not. 

In  addition  to  being  team  players,  engineering 
managers  can  further  condition  others  to  be 
receptive  to  their  views  by  establishing  a  repu¬ 
tation  for  making  reasoned  judgments.  A  pri¬ 
mary  requirement  for  establishing  such  a  repu¬ 
tation  is  that  managers  must  have  technical  ex¬ 
pertise.  They  must  be  able  to  make  technical 
judgments  grounded  in  a  sound  understanding 
of  the  principles  that  govern  science  and  tech¬ 
nology.  Systems  engineers  must  have  the  edu¬ 
cation  and  the  experience  that  justifies  confi¬ 
dence  in  their  technical  judgments.  In  the  ab¬ 
sence  of  that  kind  of  expertise,  it  is  unlikely  that 
engineering  managers  will  be  able  to  gain  the 
respect  of  those  with  whom  they  must  work. 
And  yet,  systems  engineers  cannot  be  expert  in 
all  the  areas  that  must  be  integrated  in  order  to 
create  a  successful  system.  Consequently,  sys¬ 
tems  engineers  must  recognize  the  limits  of  their 
expertise  and  seek  advise  when  those  limits  are 
reached.  And,  of  course,  systems  engineers  must 
have  built  a  reputation  for  integrity.  They  must 
have  demonstrated  a  willingness  to  make  the 
principled  stand  when  that  is  required  and  to 
make  the  tough  call,  even  when  there  are  sub¬ 
stantial  pressures  to  do  otherwise. 

Another  perhaps  small  way  that  engineers  can 
improve  communication  with  other  members  of 
their  teams  (especially  those  without  an  engi¬ 
neering  background)  is  to  have  confidence  in 
the  position  being  articulated  and  to  articulate 
the  position  concisely.  The  natural  tendency  of 
many  engineers  is  to  put  forward  their  position 
on  a  subject  along  with  all  the  facts,  figures, 
data  and  required  proofs  that  resulted  in  the  po- 

'Ethical  Issues  in  Engineering,  Johnson,  Ch  15. 


sition  being  taken.  This  sometimes  results  in 
explaining  how  a  watch  works  when  all  that  was 
asked  was  “What  time  is  it?”  Unless  demon¬ 
strated  otherwise,  team  members  will  generally 
trust  the  engineer’s  judgment  and  will  assume 
that  all  the  required  rationale  is  in  place,  with¬ 
out  having  to  see  it.  There  are  some  times  when 
it  is  appropriate  to  describe  how  the  watch 
works,  but  many  times  communication  is  en¬ 
hanced  and  time  saved  by  providing  a  confident 
and  concise  answer. 

When  systems  engineers  show  themselves  to  be 
strong  and  knowledgeable,  able  to  operate  ef¬ 
fectively  in  a  team  environment,  then  communi¬ 
cation  problems  are  unlikely  to  stand  in  the  way 
of  effective  engineering  management. 

20.2  ETHICAL  CONSIDERATIONS 

The  practice  of  engineering  exists  in  an  environ¬ 
ment  of  many  competing  interests.  Cost  and 
schedule  pressures;  changes  in  operational 
threats,  requirements,  technology,  laws,  and  poli¬ 
cies;  and  changes  in  the  emphasis  on  tailoring 
policies  in  a  common-sense  way  are  a  few  ex¬ 
amples.  These  competing  interests  are  exposed 
on  a  daily  basis  as  organizations  embrace  the 
integrated  product  and  process  development  ap¬ 
proach.  The  communication  techniques  de¬ 
scribed  earlier  in  this  chapter,  and  the  systems 
engineering  tools  described  in  earlier  chapters 
of  this  book,  provide  guidance  for  engineers  in 
effectively  advocating  the  importance  of  the  tech¬ 
nical  aspects  of  the  product  in  this  environment 
of  competing  interests. 

But,  what  do  engineers  do  when,  in  their  opin¬ 
ion,  the  integrated  team  or  its  leadership  are  not 
putting  adequate  emphasis  on  the  technical  is¬ 
sues?  This  question  becomes  especially  difficult 
in  the  cases  of  product  safety  or  when  human 
life  is  at  stake.  There  is  no  explicit  set  of  rules 
that  directs  the  individual  in  handling  issues  of 
ethical  integrity.  Ethics  is  the  responsibility  of 
everyone  on  the  integrated  team.  Engineers, 
while  clearly  the  advocate  for  the  technical  as¬ 
pects  of  the  integrated  solution,  do  not  have  a 
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special  role  as  ethical  watchdogs  because  of  their 
technical  knowledge. 

Richard  T.  De  George  in  his  article  entitled 
Ethical  Responsibilities  of  Engineers  in  Large 
Organizations:  The  Pinto  Case1  makes  the  fol¬ 
lowing  case:  “The  myth  that  ethics  has  no  place 
in  engineering  has  been  attacked,  and  at  least  in 
some  comers  of  the  engineering  profession  been 
put  to  rest.  Another  myth,  however,  is  emerging 
to  take  its  place — the  myth  of  the  engineer  as 
moral  hero.” 

This  emphasis,  De  George  believes,  is  mis¬ 
placed.  “The  zeal  of  some  preachers,  however, 
has  gone  too  far,  piling  moral  responsibility  upon 
moral  responsibility  on  the  shoulders  of  the  en¬ 
gineer.  Though  engineers  are  members  of  a  pro¬ 
fession  that  holds  public  safety  paramount,  we 
cannot  reasonably  expect  engineers  to  be  will¬ 
ing  to  sacrifice  their  jobs  each  day  for  principle 
and  to  have  a  whistle  ever  by  their  sides  ready 
to  blow  if  their  firm  strays  from  what  they  per¬ 
ceive  to  be  the  morally  right  course  of  action  ” 

What  then  is  the  responsibility  of  engineers  to 
speak  out?  De  George  suggests  as  a  mle  of  thumb 
that  engineers  and  others  in  a  large  organization 
are  morally  permitted  to  go  public  with  infor¬ 
mation  about  the  safety  of  a  product  if  the  fol¬ 
lowing  conditions  are  met: 

1 .  If  the  harm  that  will  be  done  by  the  product 
to  the  public  is  serious  and  considerable. 

2.  If  they  make  their  concerns  known  to  their 
superiors. 

3.  If,  getting  no  satisfaction  from  their  imme¬ 
diate  supervisors,  they  exhaust  the  channels 
available  within  the  operation,  including 
going  to  the  board  of  directors  (or  equiva¬ 
lent). 

De  George  believes  if  they  still  get  no  action  at 
this  point,  engineers  or  others  are  morally  permit¬ 
ted  to  make  their  concerns  public  but  not  morally 
obligated  to  do  so.  To  have  a  moral  obligation  to 


go  public  he  adds  two  additional  conditions  to 
those  above: 

4.  The  person  must  have  documented  evidence 
that  would  convince  a  reasonable,  impartial 
observer  that  his/her  view  of  the  situation  is 
correct  and  the  company  policy  wrong. 

5.  There  must  be  strong  evidence  that  making 
the  information  public  will  in  fact  prevent 
the  threatened  serious  harm. 

Most  ethical  dilemmas  in  engineering  manage¬ 
ment  can  be  traced  to  different  objectives  and 
expectations  in  the  vertical  chain  of  command. 
Higher  authority  knows  the  external  pressures 
that  impact  programs  and  tends  to  focus  on  them. 
System  engineers  know  the  realities  of  the  on¬ 
going  development  process  and  tend  to  focus  on 
the  internal  technical  process.  Unless  there  is 
communication  between  the  two,  misunderstand¬ 
ings  and  late  information  can  generate  reactive 
decisions  and  potential  ethical  dilemmas.  The 
challenge  for  system  engineers  is  to  improve 
communication  to  help  unify  objectives  and  ex¬ 
pectations.  Divisive  ethical  issues  can  be  avoided 
where  communication  is  respected  and  main¬ 
tained. 


20.3  SUMMARY 

The  material  presented  in  this  book  is  focused 
on  the  details  of  the  classic  systems  engineering 
process  and  the  role  of  the  systems  engineer  as 
the  primary  practitioner  where  the  activities  in¬ 
cluded  in  that  process  are  concerned.  The  sys¬ 
tems  engineering  process  described  has  been  used 
successfully  in  both  DoD  and  commercial  prod¬ 
uct  development  for  decades.  In  that  sense,  little 
new  or  revolutionary  material  has  been  intro¬ 
duced  in  this  text.  Rather,  we  have  tried  to  de¬ 
scribe  this  time-proven  process  at  a  level  of  de¬ 
tail  that  makes  it  logical  and  understandable  as  a 
tool  to  use  to  plan,  design,  and  develop  products 
that  must  meet  a  defined  set  of  requirements. 
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In  DoD,  systems  engineers  must  assume  roles 
of  engineering  managers  on  the  program  or 
project  assigned.  They  must  understand  that  the 
role  of  the  systems  engineer  is  necessarily  dif¬ 
ferent  from  that  normal  to  the  narrowly  special¬ 
ized  functional  engineer,  yet  it  is  also  different 
from  the  role  played  by  the  program  manager. 
In  a  sense,  the  role  of  the  systems  engineer  is  a 
delicate  one,  striving  to  balance  technical  con¬ 
cerns  with  the  real  management  pressures  deriv¬ 
ing  from  cost,  schedule,  and  policy.  The  sys¬ 
tems  engineer  is  often  the  person  in  the  middle; 
it  is  seldom  a  comfortable  position.  This  text 
has  been  aimed  at  that  individual. 

The  first  two  parts  of  the  text  were  intended  to 
first  give  the  reader  a  comprehensive  overview 
of  systems  engineering  as  a  practice  and  to  dem¬ 
onstrate  the  role  that  systems  engineering  plays 
within  the  DoD  acquisition  management  process. 
Part  2,  in  particular,  was  intended  to  provide 
relatively  detailed  insights  into  the  specific 
activities  that  make  up  the  process.  The  govern¬ 
ment  systems  engineer  may  find  him/herself 
deeply  involved  in  some  of  the  detailed  activi¬ 
ties  that  are  included  in  the  process,  while  less 
involved  in  others.  For  example,  government 
systems  engineers  may  find  themselves  very  in¬ 
volved  in  requirements  definition  and  analysis, 
but  less  directly  involved  in  design  synthesis. 
However,  the  fact  that  government  engineers  do 
not  directly  synthesize  designs  does  not  relieve 
them  from  a  responsibility  to  understand  the  pro¬ 
cess  and  to  ensure  that  sound  practices  are  pur¬ 
sued  in  reaching  design  decisions.  It  is  for  this 
reason  that  understanding  details  of  the  process 
are  critical. 

Part  3  of  the  book  is  perhaps  the  heart  of  the  text 
from  an  engineering  management  perspective. 
In  Part  3,  we  have  presented  discussions  on  a 
series  of  topics  under  the  general  heading  of 


Systems  Analysis  and  Control.  The  engine  that 
translates  requirements  into  designs  is  defined 
by  the  requirements  analysis,  functional  analy¬ 
sis  and  allocation,  and  design  synthesis  sequence 
of  activities.  Much  of  the  role  of  the  systems 
engineer  is  to  evaluate  progress,  consider  alter¬ 
natives,  and  ensure  the  product  remains  consis¬ 
tent  and  true  to  the  requirements  upon  which  the 
design  is  based.  The  tools  and  techniques  pre¬ 
sented  in  Part  3  are  the  primary  means  by  which 
a  good  engineering  management  effort  accom¬ 
plishes  these  tasks. 

Finally,  in  Part  4,  we  presented  some  of  the 
considerations  beyond  the  implementation  of  a 
disciplined  systems  engineering  process  that  the 
engineering  manager  must  consider  in  order  to 
be  successful.  Particularly  in  today’s  environ¬ 
ment  where  new  starts  are  few  and  resources 
often  limited,  the  planning  function  and  the  is¬ 
sues  associated  with  product  improvement  and 
integrated  team  management  must  move  to  the 
forefront  of  the  systems  engineer’s  thinking  from 
the  very  early  stages  of  work  on  any  system. 

This  book  has  attempted  to  summarize  the 
primary  activities  and  issues  associated  with  the 
conduct  and  management  of  technical  activities 
on  DoD  programs  and  projects.  It  was  written  to 
supplement  the  material  presented  courses  at  the 
Defense  Systems  Management  College.  The  dis¬ 
ciplined  application  of  the  principles  associated 
with  systems  engineering  has  been  recognized 
as  one  indicator  of  likely  success  in  complex 
programs.  As  always,  however,  the  key  is  for 
the  practitioner  to  be  able  to  absorb  these  funda¬ 
mental  principles  and  then  to  tailor  them  to  the 
specific  circumstances  confronted.  We  hope  that 
the  book  will  prove  useful  in  the  future  chal¬ 
lenges  that  readers  will  face  as  engineering  man¬ 
agers. 
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